Programmable edge-to-cloud virtualization fabric for the ... · Project co-funded by the European...

Project co-funded by the European Commission under the Horizon 2020 Programme.

Programmable edge-to-cloud virtualization fabric for the 5G Media industry

D3.1 - Initial Design of the 5G-MEDIA Operations and Configuration Platform

Work Package: WP3 - Operations and Configurations Framework

Lead partner: SiLO

Author(s):

Stamatia Rizou [SiLO], Takis Athanasoulis [SiLO], Francesco Iadanza [ENG], Daniele Pavia [ENG], David Breitgand [IBM], Avi Weit [IBM], George Agapiou [OTE], David Griffin [UCL], Khoa Phan [UCL], Gino Carrozzo [NXW], Francesca Moscatelli [NXW], Ugur Acar [NET], David Lopez Meco [TID]

Delivery date (DoA): 28 February, 2018

Actual delivery date: 12 December, 2018

Dissemination level: Public

Version number: 1.3 Status: Final

Grant Agreement N°: 761699

Project Acronym: 5G-MEDIA

Project Title: Programmable edge-to-cloud virtualization fabric for the 5G Media industry

Instrument: IA

Call identifier: H2020-ICT-2016-2

Topic: ICT-08-2017, 5G PPP Convergent Technologies, Strand 2: Flexible network applications

Start date of the project: June 1st, 2017

Duration: 30 months

5G-MEDIA - Grant Agreement number: 761699 D3.1 – Initial Design of the 5G-MEDIA Operations and Configuration Platform

2

Revision History Revision Date Who Description

0.1 January, 11th

2018

SiLO Document structure

0.2 January, 30th

2018

SiLO, NETAS, ENG, TID, NXW, IBM

Draft version for internal review

0.2 February, 6th 2018

ENG, NXW First round internal review

0.3 February, 22nd SiLO, ENG, UPM, IBM, NETAS, UCL

Address comments and revise contributions

0.4 February 26th ENG Final internal review

1.0 February 28th SiLO Integration of comments, finalization of deliverable

1.1 November 19th

SiLO Revision of D3.1 in line with comments received by reviewers. Ask for additional contributions.

1.2 November 30th

NXW, UCL, NCSRD

Integration of updates on 5G-MEDIA catalogue and MAPE specifications, Addition of NCSRD testbed description

1.3 December 10th

SiLO Address comments and revise contributions

Quality Control Role Date Who Approved/Comment

Internal Reviewer 6/2, 26/2 ENG Approved with minor comments addressed in the final version

Internal Reviewer 6/2 NXW Approved with minor comments addressed in the final version

Internal Reviewer 4/12 ENG, UCL Approved with minor comments addressed in the final version


3

Disclaimer This document may contain material that is copyright of certain 5G-MEDIA project beneficiaries, and may not be reproduced or copied without permission. The commercial use of any information contained in this document may require a license from the proprietor of that information. The 5G-MEDIA project is part of the European Community's Horizon 2020 Program for research and development and is as such funded by the European Commission. All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability. For the avoidance of all doubts, the European Commission has no liability with respect to this document, which is merely representing the authors’ view.

The 5G-MEDIA Consortium is the following:

Participant number Participant organisation name Short

name Country

01 ENGINEERING – INGEGNERIA INFORMATICA SPA ENG Italy

02 IBM ISRAEL - SCIENCE AND TECHNOLOGY LTD IBM Israel

03 SINGULARLOGIC ANONYMI ETAIREIA PLIROFORIAKON SYSTIMATON KAI EFARMOGON PLIROFORIKIS SiLO Greece

04 HELLENIC TELECOMMUNICATIONS ORGANIZATION S.A. - OTE AE (ORGANISMOS TILEPIKOINONION TIS ELLADOS OTE AE) OTE Greece

05 CORPORACION DE RADIO Y TELEVISION ESPANOLA SA RTVE Spain

06 UNIVERSITY COLLEGE LONDON UCL United Kingdom

07 TELEFONICA INVESTIGACION Y DESARROLLO SA TID Spain

08 UNIVERSIDAD POLITECNICA DE MADRID UPM Spain

09 INSTITUT FUER RUNDFUNKTECHNIK GMBH IRT Germany

10 NEXTWORKS NXW Italy

11 ETHNIKO KENTRO EREVNAS KAI TECHNOLOGIKIS ANAPTYXIS CERTH Greece

12 NETAS TELEKOMUNIKASYON ANONIM SIRKETI NET Turkey

13 INTERINNOV SAS IINV France

14 BITTUBES GMBH BIT Germany

15 NATIONAL CENTRE FOR SCIENTIFIC RESEARCH ‘DEMOKRITOS’ NCSRD Greece


4

Table of Contents

EXECUTIVE SUMMARY ......................................................................................................... 10

1. INTRODUCTION ............................................................................................................ 121.1. SCOPE OF THE DELIVERABLE.................................................................................................................. 12

2. OVERALL 5G-MEDIA ARCHITECTURE ............................................................................ 13

3. SPECIFICATIONS FOR 5G-MEDIA SERVICE VIRTUALIZATION PLATFORM ...................... 163.1. SPECIFICATIONS FOR OPEN SOURCE MANO ............................................................................................. 16

3.1.1 Specifications for Service Orchestrator, VNF Configuration and Abstraction and Resource Orchestrator........................................................................................................................................... 193.1.2 Specifications for Catalogues and Repositories .......................................................................... 223.1.3 Specifications for Monitoring module ........................................................................................ 243.1.4 5G-MEDIA extensions and adaptations ..................................................................................... 26

3.2. SPECIFICATIONS FOR THE MEDIA SERVICE MAPE ....................................................................................... 313.2.1 QoS/QoE Monitoring Service ..................................................................................................... 323.2.2 Cognitive Network Optimizer .................................................................................................... 333.2.3 5G-MEDIA adaptations and extensions ..................................................................................... 34

4. SPECIFICATIONS FOR 5G-MEDIA EDGE-TO-CLOUD HIERARCHY AND INSTANTIATED NFVIS/VIMS ......................................................................................................................... 37

4.1. ENGINEERING CLOUD - FIWARE PLATFORM............................................................................................. 384.2. OTE INTEGRATED INFRASTRUCTURE........................................................................................................ 39

4.2.1 VMware cloud .......................................................................................................................... 404.2.2 OpenStack cloud ....................................................................................................................... 40

4.3 NCSRD TESTBED............................................................................................................................... 434.4. TELEFONICA CLOUD – ONLIFE PLATFORM ................................................................................................. 444.5. INTEGRATION OF GPUS IN 5G-MEDIA NFVIS ......................................................................................... 46

4.5.1. Integration of GPUs into the cloud environment ........................................................................ 464.5.2. GPU provision over containerization technology ........................................................................ 48

4.6. MULTI-POP (INTER-NFVI) NETWORK CONNECTIVITY .................................................................................. 524.5.1 Multi-type VIMs ........................................................................................................................ 524.5.2 Multi-region OpenStack ............................................................................................................ 544.5.3 Multi-region OpenStack in hybrid NFVI ...................................................................................... 56

5. SPECIFICATIONS OF VIMS SUPPORTED BY 5G-MEDIA PLATFORM ............................... 57


5

5.1. INTEGRATION WITH OPENNEBULA ......................................................................................................... 575.2. FAAS VIM ARCHITECTURE AND SPECIFICATIONS ........................................................................................ 59

5.2.1 Background .............................................................................................................................. 595.2.2 FaaS VIM Architecture .............................................................................................................. 60

CONCLUSIONS ...................................................................................................................... 68

ANNEX I – INSTALLATION STEPS OF OSM V3.0 .................................................................... 69

ANNEX II – DATA MODEL FOR VNFDS IN OSM V3.0 ............................................................. 71

ANNEX III – DATA MODEL FOR NSDS IN OSM V3.0 .............................................................. 75


6

List of Figures Figure 1: Architecture of 5G-MEDIA platform ....................................................................... 14

Figure 2: Integration of OSM MANO in 5G-MEDIA architecture ............................................ 16

Figure 3: 5G Apps and Services Catalogue high level design .................................................. 23

Figure 3: Data flow of 5G-MEDIA monitoring services........................................................... 25

Figure 4: Data flow and specifications of Media Service MAPE ............................................. 32

Figure 5: Physical layer and NFVIs in 5G-MEDIA project ........................................................ 37

Figure 6: FIWARE Vicenza node in ENG cloud ....................................................................... 38

Figure 7: OTE VMWare component architecture .................................................................. 40

Figure 8: OTE Open Stack all component in one Infrastructure ............................................. 41

Figure 9: Cloud architecture that based on OpenStack ......................................................... 42

Figure 10: OTE Lab mobile infrastructure .............................................................................. 43

Figure 11: Integrated infrastructure in OTE’s lab................................................................... 43

Figure 17: NCSRD Infrastructure ........................................................................................... 44

Figure 12: Components of CORD........................................................................................... 45

Figure 13: PCI passthrough in virtual environment ............................................................... 47

Figure 14: NVidia-docker high-level architecture .................................................................. 49

Figure 15: NVidia-docker internal architecture ..................................................................... 50

Figure 16: Container orchestrators support to GPU (Source: OpenStack Summit 2017 ......... 51

Figure 17: Interconnecting multiple NFVI-PoPs ..................................................................... 53

Figure 18: Architecture of Tricircle ........................................................................................ 55

Figure 19: Kingbird architecture ........................................................................................... 56

Figure 20: Kingbird centralized deployment over different OpenStack multi-region PoPs ..... 56

Figure 21: Integration between OpenNebula and OSM ......................................................... 57

Figure 22: Plugin of OpenNebula in RO of OSM .................................................................... 58

Figure 23: Use of python-oca library for XMLRPC OpenNebula Cloud API ............................. 59

Figure 24: Reference Architecture for Integration of FaaS Frameworks with 5G-MEDIA ....... 60

Figure 25: Software architecture for FaaS integration within 5G-MEDIA Platform ................ 62

Figure 26: FaaS VNF image upload and on-going management ............................................. 63

Figure 27: Onboarded FaaS VNFD for a VNF vTranscoder being part of Star Ball Game VNS .................................................................................................................................. 64


7

Figure 28: Onboarded VNS Star Ball Game that comprises two vTranscoder FaaS VNFs interconnected by a virtual link ........................................................................................ 65

Figure 29: A User selects a VNS to instantiate ....................................................................... 65

Figure 30: OSM view of a running FaaS VNS.......................................................................... 66

Figure 31: Status of a running FaaS VNS ................................................................................ 66

List of Tables Table 2 - CLI commands in OSM ............................................................................................ 17

Table 4 - List of requests and responses through the message bus of the monitoring module ............................................................................................................................ 26

Table 5 - The list of normalized metrics supported by the OSM Monitoring module per VIM .................................................................................................................................. 26

Table 6 – List of normalized alarms supported by OSM release THREE. ................................ 28

Table 7 - Specifications of the Media Service MAPE component ........................................... 34

Table 8 - NFVI – PoP nodes ................................................................................................... 42

Table 9 - ETSI Standard: vnfd:vdu information related to hypervisors ................................... 67


8

Definitions and acronyms

AAA Authentication, Authorization, and Accounting

BSS Business Support System

CaaS Container as a Service

CO Central Office

CNO Cognitive Network Optimization

CLI Command Line Interface

CTpd Central Telefónica Procesadora de Datos

FaaS Function As A Service

GPU Graphical Processing Unit

IaaS Infrastucture as a Service

KPI Key Performance Indicator

NBI North-Bound Interface

ML Machine Learning

NFV Network Function Virtualization

NFVI Network Function Virtualization Infrastructure

NFVO Network Function Virtualization Orchestrator

NS Network Service

NSD Network Service Descriptor

NSO Network Service Orchestrator

NVML NVidia Management Library

OCP Open Compute Project

OSM Open Source MANO

OSS Operation Support System

PaaS Platform as a Service

PCI Peripheral Component Interconnect

PNF Physical Network Function

PoP Point of Presence

RBAC Role-Based Access Control

RO Resource Orchestrator


9

SBI South-Bound Interface

SDK Service Development Kit

SDN Software-Defined Networking

SLA Service Level Agreements

SO Service Orchestrator

SVF Serverless Virtual Function

SVP Service Virtualization Platform

VCA VNF Configuration and Abstraction

VDU Virtualization Deployment Unit

VF Virtual Functions

VIM Virtualized Infrastructure Manager

VLD Virtual Link Descriptors

VNF Virtual Network Function

VNFC VNF Components

VNFD Virtual Network Function Descriptor

VNFFG VNF Forwarding Graph

VoD Video on Demand

WIM WAN Infrastructure Manager


10

Executive summary As one of the most demanding 5G-enabling verticals in terms of ultra-fast transmission speed and low latency, media related applications will gain great value from the significant innovations and technical improvements in network management capabilities of 5G technologies.

The 5G-MEDIA project will extend the 5G-PPP ongoing work on NFV and SDN key technologies to offer an advanced management environment for network services and media-related applications that directly links their online lifecycle management with user experience and decisions in resources and infrastructures usage optimization. In this scope, 5G-MEDIA Operations and Configuration Platform aim to establish an open source, ETSI-NFV compliant, agile programming orchestrating and verification DevOps platform for network media services and applications, accompanied by a large set of open source network services and functions, responding to the needs of the media network challenges of H2020 according to the 5G vision.

The aim of this document is to provide the initial design and specifications of the 5G-MEDIA Operations and Configuration Platform from a technical and user experience point of view. The document builds upon the architectural specifications of the 5G-MEDIA platform carried out within Work Package 2 (“Architecture Analysis and Tools”) and represents the baseline for subsequent development activities in Work Package 3, and the rest deliverables during the project lifetime therein.

The first chapter provides an overview of the purpose of this deliverable, its relation to the rest deliverables of the project and its structure.

The second chapter introduces the initial architecture of the 5G-MEDIA ecosystem and briefly discusses the role and responsibilities of its main parts, in conformance with the KPIs that have been set by the beginning of the project with respect to the number of components integrated from other 5G-PPP projects, the accommodation of several NFVIs and VIMs, beyond those supported by the Management and Orchestration (MANO) frameworks.

The third chapter presents the technical specifications and the architectural design of the 5G-MEDIA Service Virtualization Platform (SVP), which is the central component of the 5G-MEDIA architecture. SVP leverages on European Telecommunications Standards Institute (ETSI) compliant Open Source MANO (OSM) stack to orchestrate resources and realize a multi-technology environment, where several cutting edge open source and commercial technologies, such as OpenStack, VMWare, Docker, unikernels and OpenWhisk serverless computing, are integrated with prominent software solutions of 5G-PPP phase 1 projects, such as the Service Development Kit (SDK) of SONATA, the NS/VNFs Catalogues and Repositories of SELFNET and the Monitoring-Analysis-Planning-Execution (MAPE) loop of 5G-PPP CogNet project. Beyond providing the specifications of the various components, interfaces and infrastructure to realize the 5G-MEDIA SVP, the analysis herein emphasizes especially on the extensions, adaptations and innovations which are delivered by the 5G-MEDIA project over the used enabling technologies to meet its business and use case requirements. In particular, sections 3.1.4 and 3.2.3 provide a clear view on the extensions and adaptations related to: 1) the addition of active monitoring metrics to the OSM monitoring module that are required by


11

the 5G-MEDIA use cases to support intensive networking characteristics, such as high bandwidth and low latency measurements and thus support SLA and QoE/QoS, 2) the integration of alerts related to (network and/or applicaiton) specific metrics in order to trigger actions on the NFVI, and thus optimize the performance of the running network services, 3) specific extensions to OSM related to scaling groups capabilities and VCA configuration, 4) integration of websockets to provide real-time monitoring data to Cognitive Network Optimization (CNO) components, 5) extension and adaptation of Virtual Network Function Forwarding Graph (VNFFG) characteristics supported by OSM, and 6) support of unikernels for fast deployment of microservice-based VNFs, 7) adaptation of CogNet components and tools related to MAPE, 8) extensions to Machine learning (ML) algorithms provided by CogNet or open source implementations of related ML algorithms found in the literature, in order to fulfil the requirements of the 5G-MEDIA use cases, 9) integration with SELFNET Catalogue and functionalities.

Fourth chapter presents the specifications for the hierarchical, edge-to-core cloud environment and the various NFV Infrastructures used in 5G-MEDIA project to support the considered scenarios and use cases. The technical specifications of the three cloud environments used for demonstration purposes are analyzed (in ENG, TID and OTE/NCSRD premises) and the way the 5G-MEDIA project realizes multi-NFVI network connectivity is presented. Finally, the chapter discusses how GPUs, as a technology offering one order of magnitude higher computational power compared to traditional CPUs, are integrated into the resources provided by the 5G-MEDIA cloud computing platforms to address demanding needs of media-related and entertainment services and applications.

Fifth chapter discusses the technical aspects of two main innovations delivered by the 5G-MEDIA project constituting potential contributions to the OSM community, i.e. the integration of the OpenNebula and OpenWhisk Function as a Service (FaaS) VIMs into the SVP, as enablers of integration with OnLife cloud virtualization platform and serverless computing paradigm, respectively.

Finally, the sixth and final chapter summarizes key conclusions of the deliverable.


12

1. Introduction The goal of this deliverable is to provide the initial design of the 5G-MEDIA Operations and Configuration Platform, which constitutes the backbone of the 5G-MEDIA service platform and enables the management and orchestration of media services through the implementation and integration of the MANO framework and the extension of the Service Virtualization Platform with the seamless integration with external proprietary and/or open source systems and infrastructures.

The core of the 5G-MEDIA Operations and Configuration Platform is the Service Virtualization Platform (SVP). The implementation of the 5G-MEDIA SVP will build upon the Open Source Mano (OSM) v3.0 NFV-MANO stack and will focus on the development, adaptation and extension of the following components:

• NFV MANO Service Orchestrator: this component will contain the NSD/VNFD catalogues and NS/VNF repositories customized to 5G-MEDIA project needs as well as the Network Service Orchestrator (NSO) and the VNF Manager.

• Media Service MAPE (Monitoring-Analyse-Planning-Execute): this component will realize a MAPE loop for the optimization of media services in terms of network requirements, e.g., latency, throughput, etc. The 5G-MEDIA project will focus on the implementation of the Cognitive Network Optimizer (CNO), which represents the intelligence module integrated in the MAPE loop.

• NFV MANO Resource Orchestrator: this component will contain the mechanisms for the deployment of media services on the Network Function Virtualization Infrastructures (NFVIs). The 5G-MEDIA project will provide two new Virtualized Infrastructure Manager (VIMs), in addition to those supported by OSM, i.e. the Function as a Service (FaaS) VIM, which enables the use of the so-called FaaS concept and the OpenNebula VIM, which enables the connection of OSM to OnLife NFVI provided by TID.

• Last but not least, one of the important 5G-MEDIA goals to be achieved also in this Work Package is the integration of GPUs in the 5G-MEDIA Service Virtualization Platform, providing computational power that exceeds by far the one offered by CPUs.

The 5G-MEDIA SVP will enable the connection to different NFVIs, which will host the media services of the 5G-MEDIA project. Currently, we foresee that three different NFVIs provided by the project partners (ENG, OTE/NCSRD and TID) will be used in the frame of the project. In this line, we provide an overview of the characteristics of each NFVI and a reference instantiation of the proposed 5G-MEDIA platform.

1.1. Scope of the Deliverable

This deliverable focuses on the expected outcomes of Task 3.1 “MANO Framework and NFVI Interoperability” and provides an initial design of the core 5G-MEDIA SVP components (Service Orchestrator and Resource Orchestrator). The detailed description of the Media Service MAPE is not part of this deliverable since it will be discussed in “D3.3 Specification of the 5G-MEDIA QoS Control and Management Tools” but a brief description for its scope, architectural design and specifications is also provided as reference.


13

In addition, to keep the deliverable self-contained, we include an overview on the overall 5G-MEDIA architecture. Details on the assessment of background technologies and the design of the platform architecture will be part of “D2.3 5G-MEDIA Platform Architecture”.

2. Overall 5G-MEDIA Architecture The 5G-MEDIA approach delivers an integrated programmable service platform for the development, design and operation of media applications in 5G networks. Figure 1 shows the high-level architecture of the 5G-MEDIA operations and configuration platform. The main building blocks comprising the 5G-MEDIA architecture are:

• An Application/Service Development Kit that enables access to media applications development tools, such as editor, validator, emulator, etc.

• A Service Virtualization Platform (SVP) that hosts the components related to the ETSI MANO framework1, the Virtual Network Functions and the Media Application Catalogue and Repository, the Media Service MAPE and the corresponding Virtualized Infrastructure Manager (VIM) and WAN Infrastructure Manager (WIM) plugins enabling the integration to different NFVI platforms.

• Different Network Function Virtualization Infrastructures (NFVIs) comprising the “Physical Layer” that provide computing resources by different operators and supporting different cloud technologies to host generic and media-specific VNFs and PNFs depicted at the “Virtualized Resource Layer”.

The Application/Service Development Kit (5G-MEDIA SDK) provides a set of open source tools to support the rapid development of media applications using a DevOps approach. In particular, these tools allow the definition of Media Service Forwarding Graphs (also using already existing VNFs, stored in the 5G-MEDIA VNFD/NSD Catalogue), to proof and package the various functions, as well as to emulate behaviours of the virtualized infrastructure, to accelerate application development and provide a testing environment to be utilized prior to service deployment in the runtime SVP. In addition, the 5G-MEDIA SDK tools enable the use of the innovative concept of the Function-as-a-Service (FaaS). Particularly, 5G-MEDIA pioneers in applying FaaS to VNF management, complementing traditional VM based VNFs with FaaS based media specific functions, aiming at dramatically reducing development cycles and slashing operational costs to 5G-MEDIA users. In this perspective, developers do not have to care about the low-level details related to the virtual computing and storage infrastructure (e.g. virtual server profiling in terms of CPU, RAM, etc.), thus drastically contributing to reduce the service creation time cycle and maintenance effort. In this line, the service developers will be able to create the so-called FaaS VNFs, i.e., VNFs that are instantiated upon the detection of specific events or VNFs whose events trigger specific actions on other components. The combination of the FaaS approach with the VNF packaging and the enablement of inserting FaaS VNFs in a typical VNF forwarding graph is one of the main innovation aspects of the proposed 5G-MEDIA approach.

The 5G-MEDIA SDK interacts with the Service Virtualization Platform (5G-MEDIA SVP), which hosts the components related to the ETSI MANO framework (NFV Orchestrator, VNF 1 http://www.etsi.org/technologies-clusters/technologies/nfv/open-source-mano


14

Manager(s), Infrastructure Manager(s), the VNF/NS Catalogue and Repositories etc.), as well as unique components that are used to support 5G-MEDIA internal services and deliver its applications to the external stakeholders (e.g. the Media Service MAPE).

Figure 1: Architecture of 5G-MEDIA platform


15

A major innovation of the project is the development of the Media Service MAPE component, which is composed of the Monitoring system, the Cognitive Network Optimization (CNO) engine and the Policy Manager. The development of MAPE will be based, where applicable, to the design, utilization of tools and architectural diagram being part of the outcomes of CogNet 5G-PPP phase 1 project, while achieving integration to OSM components, including Monitoring plugin and Service Orchestrator. The monitoring services aims to monitor the performance/behaviour of the instantiated Network Services (NSs), the integrated NFVIs, the interconnecting networks and the applications themselves. The measured performance metrics are directly used by the CNO engine which comprises mechanisms that take advantage of Machine Learning (ML) techniques and optimization policies to trigger the dynamic instantiation and update of VNF Forwarding Graphs (VNFFGs) or scaling options according to the capabilities provided by OSM (i.e. Scaling Groups construct in the NS descriptors) over the different NFVIs. Alternatively, after the proper execution of the policy classification procedure, the 5G-MEDIA SVP will be also able to take advantage of the benefits provided by the VCA Engine (and in particular the Juju adapter provided by OSM) to execute commands dynamically on the VNFs themselves, according to the specific use case scenario needs. As it becomes clear, the CNO is able to respond on dynamic changes of the environment (e.g., location change of end users varying QoS demands) and in cooperation with the Policy Manager to make suggestions to the MANO NFVO and VNFMs how to manage/update VNFFGs, perform scaling options, execute commands via Juju charms or even start new components enforced by the FaaS VNFs in order to meet expected QoS/SLA requirements in the most effective way. Last but not least, the QoS/QoE monitoring modules are able to provide both NSs individual and aggregated monitoring views and graphical user interfaces for metrics specified by the developer per NS/VNF, which are part of the generic monitoring system of the SVP.

In the context of 5G-MEDIA approach, as depicted in Figure 1 at the “Virtualized Resource Layer”, each media service comprises a chain of media-specific and network-specific atomic services, all interconnected to deliver an expected output to the end user (media consumer). The resulting graph (i.e., a graph where nodes refer to computing tasks and edges refer to network communication links) is referred as Media Service Forwarding Graph. During the design, development and testing phases, the 5G-MEDIA platform provides appropriate programming tools to abstract the details of the underlying 5G infrastructure and allow developers to focus on the functionality of the services. Once the media service is deployed in the virtualized infrastructure, the 5G-MEDIA platform provides mechanisms to flexibly adapt service operations to dynamic conditions and react upon events (e.g. to transparently accommodate auto-scaling of resources, VNF re-placement, etc.).

Finally, in terms of the supporting physical infrastructure (Physical Layer), the proposed architecture considers that several cloud-based NFVIs will be connected to the Service Virtualization Platform via the appropriate VIM and WIM components, enabling the use of computing and networking resources in the respective NFVI.


16

3. Specifications for 5G-MEDIA Service Virtualization Platform In this section, the architectural design of the 5G-MEDIA SVP is presented, and the specifications of its main components are identified, i.e. the NFV MANO and the MEDIA Service MAPE.

3.1. Specifications for Open Source MANO

The 5G-MEDIA SVP leverages on Open Source MANO (OSM) v3.0 to realize functionalities of the NFV-MANO stack2. In Figure 2, it is shown how OSM is integrated into the reference architecture of the 5G-MEDIA SVP. The components in purple colour, and the corresponding services, are parts of OSM stack while the components and services in blue colour are innovations, adaptations and extensions to be developed and integrated in the scope of 5G-MEDIA.

Figure 2: Integration of OSM MANO in 5G-MEDIA architecture

2 https://osm.etsi.org/


17

In 5G-MEDIA project, OSM has been installed in the core cloud of the Physical layer (see reference architecture in Figure 1) which is hosted in the ENG cloud. The installation comprises of 8vCPUs, 16GB RAM, and 80GB disk, running OS Ubuntu 16.04 (64 bit). The detailed installation steps are provided in Annex I. As a result of the installation, three LXD containers are created in the host VM, namely Resource Orchestrator (RO), VNF Configuration and Abstraction (VCA), and Service Orchestrator (SO)Open Source MANO provides a Northbound REST API3 that is intended to allow the invocation of several actions by external systems, such as an Operation Support System (OSS), SDK and the Media Service MAPE. This API supports all the operations that OSM provides and it is the unique entry point for all the interactions with the system (OSS, User Interface and Command Line Interface). The SO is in charge of triggering all actions and requests to the rest of OSM components, e.g. any action requiring direct access to VCA and RO APIs (and the internal elements and services). In this way, for instance, access to the following entities of the RO is provided4:

● Tenants: Intended to create groups of resources. In OSM v3.0 no security mechanisms are implemented.

● Datacentres: Represents the VIM information stored by OpenMano. ● VIMs: used to perform an action over a datacenter (specific pool of resources) ● VNFs: SW-based network function, composed of one or several VMs that can be

deployed on an NFV datacenter. ● Scenarios: topologies of VNFs and their interconnections. ● Instances: Each one of the Scenarios deployed in a datacenter.

The OSM Client provides a Command Line Interface (CLI) client to remotely interact with the Northbound REST API. The client gives a number of endpoints to onboard, instantiate and terminate NSs/VNFs and interact with VIMs, the most important of which are presented in Table 15.

Table 1 - CLI commands in OSM

Command Description

config-agent-add Add a new configuration agent

config-agent-delete Delete an existing configuration agent

config-agent-list List of configuration agents (i.e. OSM Juju)

ns-create Instantiate a new NS

ns-delete Delete an instantiated NS

3 OSM RELEASE THREE, A TECHNICAL OVERVIEW, Oct. 2017 [https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTHREE-FINAL.PDF], fig. 4, p.20 4 https://osm.etsi.org/wikipub/index.php/RO_Northbound_Interface 5 https://osm.etsi.org/wikipub/index.php/OsmClient


18

Command Description

ns-list List of the running NSs (in NFVI)

ns-monitoring-show Display monitoring details for an instantiated NS

ns-scale Scale NS based on the corresponding scaling group descriptor

ns-scaling-show Display details for a specific scaling group descriptor

ns-show Display details for a specific running NS

nsd-delete Delete an existing NS descriptor (included in catalogue)

nsd-list List of the NS descriptors (included in catalogue)

ro-dump

upload-package Upload a VNF package (tar.gz)

vcs-list List of internal services and their status

vim-create Register a new VIM

vim-delete Delete an existing VIM

vim-list List of the integrated VIMs

vim-show Display details for a specific integrated VIM

vnf-list List of the running VNFs (in NFVI)

vnf-monitoring-show Display monitoring details for a specific running VNF

vnf-show Display details for a specific running VNF

vnfd-delete Delete an existing VNF descriptor (included in catalogue)

vnfd-list List of the VNF descriptors (included in catalogue)

Apart from CLI, a Graphical User Interface (GUI), named Launchpad is also provided to accelerate NS design, VNFD/NSD creation as a catalogue item, VNFD/NSD on-boarding to the catalogue, VNF/NS instantiation and user management.

In the following subsections, the specifications of the 5G-MEDIA MANO are described in relation with capabilities and specifications of the OSM technology. In the last subsection, a


19

number of new extensions and adapted functionalities for OSM are summarized as they are planned to be developed in the scope of 5G-MEDIA.

3.1.1 Specifications for Service Orchestrator, VNF Configuration and Abstraction and Resource Orchestrator

The overview of 5G-Media user management and the low-level specifications of the three main modules of the 5G-MEDIA MANO (i.e. SO, VCA and RO) are provided below.

User Management

OSM Release Three (OSM-R3) has a number of advancements in terms of SVP security. Although OSM community considers security to represent a much broader set of topics, OSM-R3 has the focus of access control security and provides extensions of role-based access control (RBAC) and multi-project management.

• In terms of RBAC, the OSM SO requires a significant set of capabilities and privileges to perform all its required tasks such as VNF on-boarding, NS design on-boarding, NS deployment, NS shutdown, addition of new datacenters/VIMs, etc. However, not all of these tasks are expected to be performed by the same user or type of user in the organization. Each of the stages in the lifecycle of a VNF or NS may have different implications in terms of service continuity, validation, access to credentials, etc. OSM Release Three allows the definition of different roles, defined by admin user, with different sets of privileges. All users are mapped to at least one of these roles.

• Secondly, OSM has the concept of multi-project access control. A “project” is used to group things such as VNFDs, NSDs, NS instances, and datacenters (i.e. VIMs). The RBAC can be applied on the project definition to apply consistent access to the defined grouping of resources that OSM manages. Users have roles and can belong to more than one project. Each user is associated with one or more projects and each user has one or more roles on each project. The role is system-wide and is defined based on permissions on API endpoints. Permissions at some VIMs are restricted by the admin of the infrastructure. This is particularly true of public cloud infrastructure. In general, OSM would not have sufficient rights to create or edit tenants at VIM level. The use of the project definition enables more flexibility in grouping access to resources than if OSM allowed a VIM tenant view to permeate up the MANO stack. During the creation of OSM projects, the admin should indicate in which datacenters the new OSM project is authorized. Per datacenter, the credentials and the VIM tenant should be provided.

Media function developer, media application developer, media service developer, SVP operator are different user roles for SVP specified on D2.2. In 5G-MEDIA, the authentication and authorization of the users to specific services of the SVP are named as “User Management”. Since the OSM-R3 provides a GUI (Launchpad) for user management, new roles and specifications will be defined on further iterations adjusting the existing roles from the one provided by OSM. In the context of 5G-MEDIA, users and roles will be handled by the dedicated AAA component that will manage the configuration of the users/roles on the VIMs and MANO frameworks used in 5G-MEDIA in a transparent way from the perspective of the Catalogue.


20

OSM Service Orchestrator

In 5G-MEDIA, media applications (also called as network services) are defined as chains of one or more VNFs, the orchestration of which realizes the desirable end-to-end functionality. Each such chain is constructed and maintained by the SO module based on the information included in the corresponding Network Service Descriptor (NSD) regarding the following aspects (a detailed description of the information model defined in OSM about NSDs is provided in Annex II):

● The VNFs which are composing the service chain. ● The Physical Network Functions (PNFs), e.g. GPUs, which are used in the service chain. ● The Virtual Links Descriptors (VLDs) which describe the resource requirements needed

for links between VNFs, PNFs and the endpoints of the network service. ● The VNF forwarding graph (VNFFG) information which determines the traffic flow and

behaviour over the service chain.

Summarizing the specifications of the SO in 5G-MEDIA, this module aims at supporting the following operations:

● NS/VNF instantiation: deployment of NS/VNF instances, according to the lifecycle events defined in the corresponding NSDs/VNFDs. In 5G-MEDIA, it is possible to instantiate NSs over a hierarchical and distributed NFVI (e.g. distributed NSs), where different PoPs can be managed and operated by (at least) three different types of VIMs, i.e. OpenStack, VMWare and OpenNebula.

● NS/VNF configuration: modifications in the configuration of NSs/VNFs through their descriptors. This can be done either prior to the instantiation of a NS/VNF or as an active process while the NS/VNF is running.

● NS/VNF performance monitoring: Several different performance’s metrics from the computing instances and the network/virtual links (e.g. bandwidth, latency etc.), as they are collected by the SVP monitoring module, are provided to the SO to track critical KPIs and trigger corrective actions.

● NS/VNF scaling: increase/decrease of the NSs/VNFs instances according to the scaling policies defined per NSDs and VNFDs. This may result to creation/termination of VNF instances or updating the virtual links over them.

● VNFFG updates: update VNFFGs based on VNFFGDs and also recommendations provided by the Media Service MAPE CNO engine. This may result to re-ordering VNF list and modifying traffic routes over it.

● NS/VNF termination: release any given NS/VNF instance and its associated resources.

Service Orchestrator interfaces a Catalogue and a Repository (the specs of which are presented in §3.1.2) aiming at persistently storing information about onboarded and running VNFs and NSs, namely the public NS/VNF Catalogue and the NS/VNF Repository.

VNF Configuration and Abstraction

The VCA layer has the responsibility to enable configurations, actions and notifications to/from the VNFs and/or Element Managers. Following ETSI NFV directives, in 5G-MEDIA each VNF is composed by a set of discrete software components, called VNF Components (VNFCs),


21

where each one realizes a full description of some software functionality that is appropriately packetized as JUJU charms6, and it is onboarded at runtime into a VM image or a container. The way the VCA handles VNFs during both their instantiation and lifecycle management is described in the corresponding VNFDs, containing information with respect the following aspects:

● VNF images. ● Connectivity (connection points and virtual links), interface, and KPI requirements that

can be used by MANO functional blocks to establish appropriate virtual links, either over the VNFCs of the same VNF or between a VNF instance and the outside network and the other VNFs of the network service.

● VDUs that specify the VM/VNFC compute, storage, and network requirements. ● Platform resource requirements, such as CPU, memory, interfaces, and network. ● Special characteristics related to Enhanced Platform Awareness (EPA) attributes and

performance capabilities. ● Scaling properties.

A detailed description of the requirements and capabilities of VNF descriptors in OSM are provided in Annex III.

5G-MEDIA aims to use VCA for the configuration, and also manage JUJU charms based on REST calls instead of the usual SSH in OSM, to enable remote configuration in case of constrained scenarios such as with unikernels.

OSM Resource Orchestrator

Resource Orchestrator enables the interaction between the SO and the NFVIs by managing resources as required and interacting with multiple VIMs and SDN controllers. In OSM v.3, the RO is implemented by OpenMano and it has already been integrated with OpenStack, VMware, OpenVim and Amazon Web Services (AWS) VIMs. 5G-MEDIA aims to extend several RO functionalities of OSM to support the following operations:

● Multi-PoP deployment: the RO would be able to deploy the VNFs across different PoPs and domains, controlled either by the same or different NFVIs/VIMs.

● PNF integration: ability to deploy a NS that requires the deployment of one or more network functions on physical machines (i.e. GPUs).

● Support of multiple virtualizations technologies: support the deployment of VNFs on multiple virtualization technologies, including VMs, containers and unikernels.

● Computational resilience: automatic re-provisioning of resources over server failure or network disruption.

● Serverless support: deploy, run and manage VNFs packetized under the FaaS computing model.

6 https://osm.etsi.org/wikipub/index.php/Creating_your_own_VNF_charm_(Release_THREE)


22

3.1.2 Specifications for Catalogues and Repositories

In 5G-MEDIA, the Catalogue is an architecture building block that operates on top of the MANO stack with the primary goal of validating, versioning and storing different kinds of 5G application and service descriptors/packages, i.e. NSDs/PNFDs, VNF Packages but also SDN App Packages.

The design of the 5G-MEDIA Catalogue started with an analysis of available opensource tools and solutions providing NFV catalogue features with the aim of identifying a suitable reference implementation to be adopted and properly enhanced for the purpose of the 5G-MEDIA 5G Apps and Services Catalogue. In particular, this state-of-the-art analysis included the investigation of the catalogue components and features offered by Open Source NFVOs (e.g. OpenSourceMANO7 and OpenBaton8) and the SELFNET App Catalogue9, developed in the context of the 5GPPP H2020 phase 1 project SELFNET.

One of the 5G-MEDIA primary goals is to develop a generic 5G Apps and Services catalogue for storing and managing a heterogeneous set of descriptors and packages for NFV Network Services, VNFs, PNFs and SDN applications, agnostic of the specific NFV and SDN tools information models, and aligned with the latest ETSI specifications for NSDs and VNF Packages management, i.e. ETSI GS NFV-SOL005 Os-Ma-Nfvo10.

At the time of writing, then taking into consideration OSM-R3 and OpenBaton R5, the state-of-art analysis found that OSM-R3 offers no ETSI standard-compliant information model for the representation of descriptors and packages (e.g. aligned with ETSI GS NFV-SOL00111 and SOL00412) neither a standard aligned NBI, while OpenBaton R5 offers an ETSI GS NFV-MAN00113 compliant information model, then not able of representing all the VNFD and NSD information elements defined respectively in ETSI GS NFV-IFA01114 and IFA01415. On the other hand, the SELFNET App Catalogue provides a full on-boarding service based on a common and unified information model for describing VNFs, PNFs and SDN Apps capabilities in terms of deployment, configuration and monitoring requirements and features.

7 https://osm.etsi.org/ 8 https://openbaton.github.io/ 9 https://bscw.selfnet-5g.eu/pub/bscw.cgi/d74819/D3.1-Report%20and%20Prototype%20Implementation%20of%20the%20NFV%20%26%20SDN%20Repository.pdf 10 https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/005/02.05.01_60/gs_NFV-SOL005v020501p.pdf 11 https://docbox.etsi.org/ISG/NFV/Open/Drafts/SOL001_TOSCA_desc 12 https://www.etsi.org/deliver/etsi_gs/NFV-SOL/001_099/004/02.05.01_60/gs_NFV-SOL004v020501p.pdf 13 https://www.etsi.org/deliver/etsi_gs/NFV-MAN/001_099/001/01.01.01_60/gs_NFV-MAN001v010101p.pdf 14 https://www.etsi.org/deliver/etsi_gs/NFV-IFA/001_099/011/02.05.01_60/gs_NFV-IFA011v020501p.pdf 15 https://www.etsi.org/deliver/etsi_gs/NFV-IFA/001_099/014/02.05.01_60/gs_NFV-IFA014v020501p.pdf


23

Figure 3: 5G Apps and Services Catalogue high level design

Since no existing solution is aligned with latest or, at least, recent ETSI standards in terms of offered NBI and descriptors’ information elements representation, the SELFNET App Catalogue is selected as the baseline for the implementation of the 5G Apps and Services Catalogue. As the other two candidates, the SELFNET App Catalogue does not offer an ETSI SOL005 aligned NBI neither an ETSI SOL001/SOL004 compliant information model, but on the other hand we identify its key and unique characteristics/features to be leveraged and enhanced in the development of the 5G Apps and Services Catalogue, as follows:

• A common information model for representing different types of Apps (VNF, PNF, SDN).

• A Plugin based approach for decoupling and being independent of underlaying tools (NFV MANO, SDN controller) APIs and information models.

• A common one-click on-boarding for VNF Packages and SDN Apps in the form of software archives.

5G-MEDIA re-architects the SELFNET App Catalogue to have a full compliancy with ETSI NFV SOL specifications for NSDs and VNF Packages management (i.e. SOL005), as well as TOSCA information models for representing NSDs/PNFDs, VNFDs and VNF Packages (i.e. SOL001 and SOL004). This allows to have a complete on-boarding tool compliant with latest ETSI NFV specifications, hiding at the same time the complexity of specific and technology dependent MANO models and interfaces, while allowing developers and service providers to implement a portable offer of applications that it’s not strictly bound to a specific MANO product.


24

The main features of the SELFNET App Catalogue that are kept and still enhanced in the 5G-MEDIA Catalogue implementation are:

• Plugin based approach for being agnostic of information models and APIs of NFV MANO tools and monitoring subsystems. The 5G Apps and Services Catalogue plugin-driven design is highlighted in Figure 3. In particular, with respect to the SELFNET App Catalogue, the new 5G-MEDIA Catalogue provides:

o Runtime plugins configuration capabilities for enabling and disabling plugin instances.

o MANO plugins for on-boarding NSDs, PNFDs and VNFDs/VNF Packages into the targeted NFVO inner catalogue (e.g. OSM catalogue). NSDs were not supported and MANO plugins were not implemented in SELFNET

o Translation functions in each MANO plugin to translate the generic descriptor format into the specific one expected at the plugin’s targeted MANO stack.

The Notification Dispatch interface implementation follows the SELFNET design, with the functional scope of interconnecting plugins with the core and the NBI handlers and dispatching monitoring info to the monitoring subsystem, i.e. the 5G-MEDIA MAPE.

• Modelling of monitoring metrics offered by applications/functions as clear separated info in the onboarded archive. The 5G-MEDIA catalogue will follow the SELFNET approach for describing application level metrics to be consumed by the 5G-MEDIA MAPE. In particular, in 5G-MEDIA, the monitoring descriptor, once onboarded into the Catalogue along with the rest of the package, will be dispatched to the 5G-MEDIA MAPE with the preliminary intention of setting-up the environment for the service/app to be instantiated and monitored. The original monitoring descriptor format from SELFNET will be adapted in order to comply with the pre-configurations needed at the 5G-MEDIA MAPE (e.g. the configuration of a specific topic into the SVP bus).

3.1.3 Specifications for Monitoring module

In October 2017, the latest release of OSM has been announced, providing many features missing from the previous releases. One of them was the addition of the Monitoring module16 (although in experimental version). Based on the decision of OSM Community not to develop and implement another monitoring solution, OSM provides interfaces to already existing monitoring tools for all supported VIMs, namely OpenStack, VMWare and AWS. In this perspective, OSM Monitoring module is not intended to replicate or compete with these systems, but to provide the means for easy integration under a commonly agreed data model.

In the current release, OSM supports a flexible, plugin-based method for integrating several external (existing) monitoring tools. In particular, it provides plugins for Aodh and Gnocchi for OpenStack, vRealize Operations Manager for VMWare and CloudWatch for AWS. On top of the plugins, a well-defined Information Model parses incoming data and represents them in

16 https://osm.etsi.org/wikipub/index.php/OSM_MON_Module_Installation_Guide_(Release_THREE)


25

the format accepted by the internal modules of OSM, e.g. SO and/or VCA while communicating with the Monitoring module.

The data flow of 5G-MEDIA monitoring services is shown in Figure 4. The OSM Monitoring module executes two main functions: the first one is its ability to interact with the external monitoring tool in order to perform configuration updates related to monitoring metrics and alerting rules. The second one is the ability to steer actionable events to the Service Orchestrator. Moreover, the monitoring module is capable of correlating monitoring data related to the VNFs that consist a NS and thus provides an improved user experience compared to other solutions. It is highlighted that the module includes an instance of an Apache Kafka that is used as the message bus for the communication of all sub-modules of Monitoring. Kafka provides a publish-subscribe model, based on topics that interested users subscribe to.

Figure 4: Data flow of 5G-MEDIA monitoring services

The generic procedure followed for the message exchange is the following: the SO sends a request to the OSM monitoring module through Apache Kafka message bus. The plugin consumer reads the messages and talks to the appropriate VIM monitoring tools to configure them according to the encapsulated command in JSON format. In this release, the alarms and metrics have dedicated topics in the message bus, as shown in Table 2. Each message is sent on the message bus in JSON format, along with a unique request key and its topic.


26

Table 2 - List of requests and responses through the message bus of the monitoring module

Request & Response Description

create_alarm Message through Kafka topic to create alarm

create_metric Message through Kafka topic to create metric

list_alarm Message through Kafka topic to list alarm

list_metric Message through Kafka topic to list metric

delete_alarm Message through Kafka topic to delete alarm

delete_metric Message through Kafka topic to delete metric

update_alarm Message through Kafka topic to update alarm

update_metric Message through Kafka topic to update metric

acknowledge_alarm Message through Kafka topic to acknowledge alarm

acknowledge_metric Message through Kafka topic to acknowledge metric

3.1.4 5G-MEDIA extensions and adaptations

Extensions with respect to monitoring

Due to the fact that release three is the first OSM release where monitoring is supported (in experimental state), there are several adaptations and possible extensions that have to be considered for development and implementation in order to fulfil the Use Case requirements of 5G-MEDIA. First of all, the number as well as the “nature” of metrics supported at this version is neither large nor covering the demands that could be set by the majority of the use cases in the future, including those that 5G-MEDIA project is focusing on. More specifically, Table 3 depicts the supported metrics per VIM, along with a short description and the respective VIM.

Table 3 - The list of normalized metrics supported by the OSM Monitoring module per VIM17

Normalized Metric Name

Unit VMware vROPs Metric

Amazon CloudWatch Metric

OpenStack Metric

AVERAGE_MEMORY_UTILIZATION

% mem|usage_average

Not Supported Memory.total (%)

17 https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTHREE-FINAL.PDF


27

Normalized Metric Name

Unit VMware vROPs Metric


OpenStack Metric

READ_LATENCY_<DISK_NO>

msec

virtualDisk|totalReadLatency_average

Not Supported Not Supported

WRITE_LATENCY_<DISK_NO>

msec

virtualDisk|totalWriteLatency_average

Not Supported Not Supported

DISK_READ_OPS

Nos

Not Supported

DiskReadOps

Disk_ops.DISK

DISK_WRITE_OPS Nos Not Supported DiskReadOps Disk_ops.DISK DISK_READ_BYTES

Bytes or bytes/sec

Not Supported

DiskReadBytes

Disk_octets.DISK

DISK_WRITE_BYTES

Bytes or bytes/sec

Not Supported

DiskReadBytes

Disk_octets.DISK

PACKETS_DROPPED_<NIC_NO>

Nos

net|dropped

Not Supported

if_dropped.INTERFACE

PACKETS_RECEIVED

Nos

net:Aggregate of all instances|packetsRxPerSec

NetworkPacketsIn if_packets.INTERFACE

PACKETS_SENT

Nos

net:Aggregate of all instances|packetsTxPerSec

NetworkPacketsOut

If_packets.INTERFACE

CPU_UTILIZATION

% cpu|usage_aver

age

CPUUtilization

Percent.virt_cpu_total

It is obvious that the list of Table 3 lacks support for networking monitoring metrics, such as bandwidth, latency, packet loss, capacity, etc. This is critical for 5G-MEDIA, since the scenarios


28

are mostly based on bandwidth and latency requirements, while QoS is supposed to be supported through a Graphical User Interface. In this respect, 5G-MEDIA will extend this list by providing the means to include measurements from sources other than those defined in the external monitoring tools supported by OSM currently. This requirement covers both passive and active monitoring measurements within the 5G-MEDIA NFVI under question. Active monitoring deploys probes and transmits packets into the network between two endpoints. Commonly used tools include ping, which measures delay and packet loss, as well as traceroute, which helps to determine topology of the network. The main problem with active monitoring is that by the deployment of monitoring probes and the injection of packets, interference to normal traffic can be observed. On the other hand, active network monitoring allows full control over additional packets that are required to be sent over the network, as they can be sent whenever required by any specific monitoring application hence are more flexible. Unlike active monitoring, passive monitoring does not inject packets in the network but rather collects information about one point in the network that is being measured rather than between two endpoints, as it is the case on active monitoring concept. Passive monitoring can be achieved by the utilization of any sniffing application and the offline processing of collected data. As a conclusion, passive monitoring is helpful when network administrator needs to know details such as network topology, running services, ports used frequently, etc. This is, in most of the cases, achieved by scanning the TCP and IP headers using various packet sniffers (e.g. tcpdump). Utilizing the advantages of both monitoring methodologies, 5G-MEDIA will extend the list of supported metrics and fully cover the requirements posed by the use case scenarios, implementing well-known, open source active monitoring tools such as iPerf to cover specific needs where required.

Moreover, as shown in Table 4, the list of alarms does not cover the demands of 5G-MEDIA and in general the developers’ and service providers’ needs with respect the performance monitoring of the network service and the application on top of it as deployed in the 5G-MEDIA infrastructure. The extensions foreseen in 5G-MEDIA complement the monitored metrics that might be reflected by the definition of an alarm to trigger specific actions and inform the respective users (infrastructure owner, service developer, service provider).

Table 4 – List of normalized alarms supported by OSM release THREE.

Normalized Alarm Name

VMware vROPs Metric


OpenStack Metric

Average_Memory_Usage_Above_Threshold


Not Supported


Read_Latency_Above_Threshold

Read_Latency_Above_Threshold

Not Supported

Not Supported

Write_Latency_Above_Threshold

Write_Latency_Above_Threshold

Not Supported

Not Supported


29

Normalized Alarm Name

VMware vROPs Metric


OpenStack Metric

Disk_read_ops_above_threshold

Not Supported

Disk_read_ops_above_threshold

Disk_ops.DISK

Disk_write_ops_above_threshold

Not Supported

Disk_write_ops_above_threshold

Disk_ops.DISK

Disk_read_bytes_above_threshold

Not Supported

Disk_read_bytes_above_threshold

Disk_octets.DISK

Disk_write_bytes_above_threshold

Not Supported

Disk_write_bytes_above_threshold

Disk_octets.DISK

Net_Packets_Dropped

Net_Packets_Dropped

Not Supported

Net_Packets_Dropped

Packets_in_Above_Threshold



if_packets.INTERFACE

Packets_out_Above_Threshold



if_packets.INTERFACE

CPU_Utilization_Above_Threshold




Close the DevOps loop through utilization of monitoring data

Another extension related to monitoring is based on the shortcoming of OSM to implement the northbound interface of the Monitoring module18. This has a direct impact on “closing the cycle” between Development and Operations (DevOps) in the respect that data (and respective alarms) coming to the Monitoring module from external monitoring tools are not

18 https://osm.etsi.org/images/OSM-Whitepaper-TechContent-ReleaseTHREE-FINAL.PDF


30

consumed by any actors/modules. Thus, OSM functionalities such as Scaling Groups and VCA charms remain underutilized. The Scaling Groups construct within the NSD can be used to identify a group of VNFs (sharing common characteristics) that need to be scaled under certain performance (or fault) conditions. In this release, the scaling actions supported include scaling in and out; in the case of scaling out, the added VNFs instances are attached to the same network as the initial VNF instances while the scaling action is triggered manually using the OSM GUI or the Northbound API. The conditions that trigger actions are derived from system metrics collected by the Monitoring module and thus a better support for such cases is provided in the 5G-MEDIA SVP. The CNO component of the MAPE system employs novel ML-based algorithms as well as classical optimization techniques to automatically process monitored data to identify the conditions for triggering actions. Hence, execution is automated, based on the results of the ML classification processes or optimization algorithm results. Supervised and reinforcement machine learning approaches, genetic algorithms and classical mixed integer linear programming are currently being validated in the context of the three 5G-MEDIA use cases. Additionally, the VNF Configuration and Abstraction (VCA) layer supports configuration updates, actions and notifications to/from already running instances of VNFs. Backed-up by interfaces to trigger these actions, VNFs can be updated during runtime, providing several practical advantages during use case testing and demonstration execution.

Providing streaming monitoring data over websockets technology

During the study of the methodologies and technologies that have been used by 5G-PPP Phase 1 projects, it has been discovered that SONATA monitoring framework has implemented a streaming “channel” over websockets technology to provide a (near) real-time stream of monitoring data to interested user whose services are running on the 5G-MEDIA distributed infrastructure. The concept is based on the instantiation of a Tornado server that creates websockets that are passed to previously authenticated users of the SONATA Service Platform. Apart from the obvious advantage compared to the provisioning of an API call, there are two interesting things that 5G-MEDIA would like to further investigate and probably build upon: the need for streaming data might be proved really useful to developers, having the need to check the service parameters in real-time. The second remark to be taken into consideration is the introduction of the CNO in the 5G-MEDIA architecture. In the case that real-time data need to be consumed, websockets could be an alternative to the utilization of a message bus that has been implemented in CogNet and that 5G-MEDIA will leverage.

Providing VNF Forwarding Graphs capabilities for dynamic modifications

One of the most important characteristics of the MANO frameworks that have been considered for adoption in 5G-MEDIA was related to their ability to support the definition and enforcement of forwarding graphs between VNFs consisting network services. This feature has been further highlighted in 5G-MEDIA, since one of the main objectives of Cognitive Network Optimizer is the ability to provide recommendations to the operator in order to optimize the use of resources through VNF placement. By investigating offerings from these MANO frameworks, it became evident that none of these solutions could be used “as-is”,


31

and it was clear that adaptations or even extensions to current approaches would be required. In OSM, a Virtual Network Function Forwarding Graph (VNFFG) is considered a deployment template that defines the network service topology and end-to-end traffic behaviour. It is defined in NS’s descriptor file and orchestrated by the NFVO. The VNFFG may cover virtual links, VNFs, and NFs. A VNFFG is a graph, specified by a network service provider, of bi-directional logical links that connect network function nodes, where at least one node is a VNF through which network traffic is directed. The VNFFG model is defined at the network service level, where one or more VNFFG descriptors can be defined in the respective descriptor (NSD).

Handling changes on VNF forwarding graph level in a dynamic way is of paramount importance for the project. However, dealing with dynamic re-configuration of VNFs is not a trivial task, while in some cases might even be impossible to be realized without loss of the service at least for a period of time.

Extensions with respect to unikernel

Unikernels are specialised virtual machine images constructed by compiling high-level languages that run directly on a hypervisor 19. They provide only the strictly necessary code and, as a consequence, they have an improved security, smaller footprint and shorter boot time compared to traditional operating systems on virtual machines. Unikernels in 5G-MEDIA project will be used to support the development of application layer services (e.g. streaming service) and support the adoption of microservices architecture20. 5G-MEDIA project will integrate the packaging tools (such as "capstan21”) provided by Mikelangelo research project22 to generate specialised machine images hosting Java, Javascript or Python microservices running on OSv unikernel images23. Such machine images will be fully integrated in the CI/CD pipeline and be referred by the VNF descriptor in a seamless manner and they will support OpenStack24 using KVM hypervisor 25.

3.2. Specifications for the Media Service MAPE

The 5G-MEDIA Media Service Monitoring-Analysis-Planning and Execution (MAPE) component implements an automation mechanism aiming at assisting MANO to dynamically manage and provide infrastructure resources according to the changes in user demand patterns, and the availability and performance of network and computational resources. The MAPE execution loop consists of the following four concrete sequential steps of activities:

19 http://unikernel.org/ 20 https://smartbear.com/learn/api-design/what-are-microservices/ 21 https://github.com/mikelangelo-project/capstan-packages 22 https://www.mikelangelo-project.eu/ 23 https://github.com/mikelangelo-project/osv 24 https://www.openstack.org/ 25 https://www.linux-kvm.org/page/Main_Page


32

1) Monitor: Collect data from integrated infrastructure and network elements used to monitor network conditions, resources performance and services status and enable the analysis step.

2) Analyse: Process collected data by using machine learning to improve specific performance KPIs of individual and aggregated network services and the network environment.

3) Plan: Recommend decisions and output concrete control actions aiming to mitigate certain risks with respect to the network infrastructure and the provided services.

4) Execute: Execute commands upon available resources and infrastructure by invoking the MANO stack and its modules according to the recommended actions.

The software architectural design and data flow of the 5G-MEDIA MAPE are shown in Figure 5.

Figure 5: Data flow and specifications of Media Service MAPE

As shown in Figure 5, the 5G-MEDIA Media Service MAPE is realized on top of two main sub-components, namely the QoS/QoE Monitoring Service and the Cognitive Network Optimizer (CNO), which implement all the services introduced by the MAPE chain. The specifications of these two sub-components are provided below:

3.2.1 QoS/QoE Monitoring Service

The QoS/QoE monitoring service collects data from running NSs/VNFs and the SDN controller. A detailed list of the available monitoring data is provided in section §0. Among other, the collected data include %CPU, %RAM, BW in-out, latency, packets drop/received/send etc.

The QoS/QoE monitoring services are responsible to i) retrieve data from OpenStack, VMWare and OpenNebula NFVIs, ii) implement a data modelling/indexing schema that enables flexible management and analysis of the collected data, iii) permanently store data in a metrics


33

database which is accessible by the ML and optimization algorithms of the CNO, and iv) share data to the CNO in real-time through a high performance, distributed, and scalable message queue based on Apache Kafka (to enable near real-time analysis capabilities).

3.2.2 Cognitive Network Optimizer

Cognitive Network Optimizer (CNO) is an analysis and prediction engine that uses the raw measurement data from the QoS/QoE monitoring service and employs statistical analysis based on machine learning and optimization techniques for classifying network and computational resource service status, predicting/forecasting future resource conditions, and triggering corrective actions for the running NSs.

The CNO consists of two main sub-components; the first is the Machine Learning Engine and the second the Policy/Optimization Engine. The Machine Learning Engine builds upon the existing specifications and open source software developed by the CogNet 5G-PPP phase 1 project:26 the CogNet Lightweight Smart Engine. 5G-MEDIA’s Machine Learning Engine groups together a set of Machine Learning (ML) modules quantifying network status and providing services, such as (big) data dimensionality reduction, feature selection/extraction, traffic classification, anomaly detection, performance degradation detection and demand prediction. It is used to analyse the behaviour of the instantiated network services (e.g. classify traffic, predict resource demand and utilisation, evaluate SLA violation risks, etc.) to deliver insights to the Policy/Optimization Engine on both current performance metrics and forecasted changes in demand and expected performance levels. The inputs from the Machine Learning Engine provide the necessary inputs to the multi-objective optimization algorithms that will determine the assignment of the available network and computational resources to maximizing performance metrics within defined cost and efficiency constraints.

Table 5 presents an initial set of ML modules that have been identified to assist in the optimization of 5G-MEDIA scenarios and use cases. The Machine Learning Engine applies both to: stream processing algorithms, i.e. applied on top of data streams provided by QoS/QoE monitoring services in near real-time scale; and batch processing algorithms, i.e. applied over new and historical data (as they are retrieved by accessing the metrics DB) to identify possible time/space and events-related correlations in an offline way. The output of the Machine Learning Engine is passed after the necessary adaptation phase as an active indication to the Policy/Optimization Engine. The latter aims at applying the most appropriate algorithms to optimize usage of infrastructure resources and performance of instantiated NSs, under the constraints set by the service providers and the network administrator (e.g. wrt latency, maximum resource utilization, budget cost, etc.), both in individual and aggregated levels. For instance, Integer Linear Programming can be used over small input datasets related to a specific NSs (or a suitable heuristic algorithm can be employed for larger input datasets) to determine when is the optimal time to scale in/out its VNF-FG by removing/adding further resources. By interfacing to the northbound API of the SO, the output of the

26 http://www.cognet.5g-ppp.eu/


34

Policy/Optimization Engine is conveyed to the MANO stack which will enforce the recommended policies over the resources of the integrated NFVIs.

3.2.3 5G-MEDIA adaptations and extensions

The extensions and adaptations foreseen for 5G-MEDIA are partially related to the work that has been conducted in CogNet with respect to the testing infrastructure, the ML tools as well as the algorithms and their adaptation to fulfil the specific 5G-MEDIA use case requirements. As discussed below, the first contribution of 5G-MEDIA is related to the adaptation of the CNO with Ceilometer, Gnocchi and Aodh, being the monitoring tools utilized by OSM, replacing Monasca that it was the selected tool of CogNet for collecting monitoring data from OpenStack environment. As a second extension, 5G-MEDIA plans to capitalize on the ML algorithms that will become available from CogNet after the end of the project. 5G-MEDIA consortium has already provided a first filtering of the algorithms that could potentially be used and extended for the purposes of the use case scenarios involved in the project, as mentioned in the following Table. Moreover, also in the Table below, the consortium has classified the four main types of algorithms for the optimization of media services under consideration in the three 5G-MEDIA use cases, to find similarities and differences with CogNet.

In Table 5, we summarize the specifications of the Media Service MAPE sub-components. Further details on the technical specifications for this component are contained in D3.3. The final design of the algorithms developed in the context of 5G-MEDIA will be contained in D3.4.

Table 5 - Specifications of the Media Service MAPE component

Sub-component Specifications

QoS/QoE monitoring 1) Data are collected by consuming/exposing interfaces from/to: a. Ceilometer, Gnocchi and Aodh to collect data from

OpenStack NFVIs, b. vRealize to collect data from VMware NFVIs, c. the monitoring module of OnLife NFVIs

2) The Metrics DB is based on a time series Influx DB. Depending on the needs of the considered scenarios, distributed deployments (clustering) will be explored.

3) Real-time data sharing between QoS/QoE monitoring services and CNO will be served by an Apache Kafka.


35


CNO Machine Learning Engine

ML modules, partially derived from the CogNet project27 and adapted for 5G-MEDIA use case scenarios, include:

1) Mobile User Throughput Prediction (ML4MQ): detection of SLA breaches at service level.

2) Anomaly Detection Engine (ADE) with Long Short Term Memory Neural Networks: detection of resource performance degradation.

3) Network Traffic classification (NTC): multi-classification model of NSs based on labelled network-traffic KPI data.

4) Distributed Application Performance Optimizer (DAPO): classifier of network load and predictor of critical KPIs

ML modules are encapsulated in Docker containers and process metrics from InfluxDB and the Apache Kafka message bus using the machine learning libraries provided by Apache Spark, Google TensorFlow/Keras and Eclipse Deeplearning4j to produce outputs.

27 http://www.cognet.5g-ppp.eu/wp-content/uploads/2018/01/D34_version10_final.pdf


36


Policy/Optimization Engine

The Policy/Optimization Engine takes input from the Machine Learning Engine and from operator-defined policies (e.g. maximum cost budget and lowest acceptable performance level to meet SLA objectives) about the status of the network resources and the instantiated NSs and executes its optimization algorithms. The output of which are implemented as policy directives to be executed by the SO. The communication with the northbound API of the OSM SO is through JSON data models and the RESTful architectural style.

Algorithms for the optimization of media services under consideration in the three 5G-MEDIA use cases are of four main types:

• Service placement optimization to determine which VNFI instance/edge node should house each VNF for a NS by trading-off cost with performance of the network and computational infrastructure. This can run at various timescales, including initial NS deployment and ongoing reconfiguration to migrate existing VNFs, instantiate new VNFs, undertake service scaling as demand patterns change.

• VNFFG optimization to determine which instances of VNFs should be interconnected to meet performance and cost objectives for specific user session requests. This can be undertaken at initial session establishment as well as for the optimization of already running sessions/VNFFG instances.

• Infrastructure adaptation to overcome streaming difficulties, e.g. to reserve network capacity, allocate greater computational capacity for stream processing, establish expedited paths or reroute flows to avoid congested parts of the network.

• Application-specific adaptation and intelligent network-wide congestion avoidance, for example to configure the capturing or transcoding of 3D models to defined quality levels to match dynamically varying network throughput capabilities and available processing capacity along the VNFI nodes and clusters implementing the VNFFG instance.

5G-MEDIA has adopted the CogNet results in terms of the architectural concepts and the choices for the software environment for incorporating ML algorithms. The added-value of 5G-MEDIA compared to CogNet is on:


37

• The development of specific ML techniques to be used for the optimization and automated configuration of media application in the three project use cases. Supervised and reinforcement learning approaches are being adopted for anomaly detection, traffic prediction and for using QoE feedback as a reward/penalty function for adjusting the allocation of resources to the application VNFs as well as application-specific actions such as compression levels.

• The incorporation of additional ML libraries into the suite of tools available to 5G-MEDIA: including TensorFlow/Keras and Deeplearning4j.

• Integration of 5G-MEDIA ML algorithms with the 5G-MEDIA monitoring system through the consumption of Kafka topics tailored to the 5G-MEDIA SVP workflow and the specific environments being monitored (OpenStack and OpenNebula computational environments, application monitoring through the use of Telegraf and the network resources through access to ONOS SDN controller API and IxChariot performance testing tools.

• Integration of ML algorithms with classical optimization techniques, using mixed integer linear programming in IBM’s CPLEX, for example.

• Integration of the output of the ML algorithms with the execution services of the 5G-MEDIA MAPE component, with specific actions tailored for the OSM SO, the OpenStack and OpenNebula NFVIs, the FaaS VIM and to application-specific VNFs.

4. Specifications for 5G-MEDIA edge-to-cloud hierarchy and instantiated NFVIs/VIMs

The 5G-MEDIA NFVI, shown in Figure 6, is composed of three different cloud sites geographically distributed and belonging to different organization, i.e. ENG in Italy, TID in Spain and OTE in Greece). The core cloud, hosting SVP and also an OpenStack NFVI providing compute, storage and network resources for NSs and VNFs, is provided by the ENG cloud, while other two cloud edge environments are provided by TID and OTE to support the various considered scenarios and use cases of the project. The networking interconnection and communication between the core and the two edge cloud environments will be achieved by using HTTPS connections, while other security features could be taken into account (i.e. limit the source address to those used in ENG) depending on the admin policies in OTE/TID. In the following subsections, the specifications and other technical aspects of the three environments are analysed.

Figure 6: Physical layer and NFVIs in 5G-MEDIA project


38

4.1. Engineering cloud - FIWARE Platform

The core network cloud of 5G-MEDIA project is hosted in the FIWARE platform premises of ENG. From a technical standpoint, the FIWARE Platform is an OpenStack-based federated infrastructure comprised of several nodes, with some peculiar aspects:

● a Keystone28 based centralized Authentication, Authorization and Accounting service shared across all the FIWARE nodes

● a Ceilometer29 and Monasca30 based centralized monitoring service shared across all the FIWARE nodes, which constitute the pillar above which the infographics31 that show the overall status of the FIWARE platform has been built.

● daily sanity checks are performed on every node of the platform ● a customized, Horizon32 based user interface ● a wide Generic Enablers offering available through a software catalogue33 for rapid and

easy deployment of software architecture blocks Regarding the 5G-MEDIA project, Engineering Ingegneria Informatica s.p.a. is fielding its FIWARE Vicenza node, shown in Figure 7.

Figure 7: FIWARE Vicenza node in ENG cloud

28 Keystone, the OpenStack Identity Service: https://docs.openstack.org/keystone/latest/ 29 Ceilometer’s documentation: https://docs.openstack.org/ceilometer/latest/ 30 Monasca: https://wiki.openstack.org/wiki/Monasca 31 FIWARE Lab Nodes: http://infographic.lab.fiware.org/ 32 Horizon: The OpenStack Dashboard Project: https://docs.openstack.org/horizon/latest/ 33 Generic Enablers FIWARE Catalogue: https://catalogue.fiware.org/enablers


39

The FIWARE Vicenza node is based on the OpenStack Kilo release, however an update to a newer OpenStack version has been scheduled across all FIWARE nodes in the upcoming months as well as the provisioning a new, dedicated instance of OpenStack to the project. Despite FIWARE user interface customizations, this deployment is fully compliant with the RESTful APIs offered by a standard OpenStack installation (please refer to the official OpenStack API Documentation34 for more info). The node is comprised of three redundant, highly available controller nodes, five compute nodes and two CEPH35 storage nodes. In terms of available resources, here is a quick recap:

• 432 vCores • 384GB of RAM • 16TB of NAS/SAN disk space, plus 300GB of local disk space for each node • 8 10Gb interfaces per node, connected to two 10Gb switches

Moreover, the hosts network interfaces are bound in couples to achieve high networking availability. Four virtual LANs are employed to isolate different kinds of network traffic, such as internal server, internal Virtual Machine, public, and administrative data. Two firewall layers and port filtering techniques are leveraged to address network security. Common port ranges used in typical network services such as HTTP, HTTPS, SSH, FTP and others may be freely bound on any instantiated Virtual Machine that has a public floating IP address. Two mirrored CEPH storage nodes offer object, block and file storage to the virtualization infrastructure. The FIWARE node infrastructure is hosted at a tier 4 Data Centre, thus providing 99.995% uptime, 96 hours power outage protection and fully redundant network, power and cooling infrastructures.

In the context of the 5G-MEDIA project, the Vicenza FIWARE node is hosting several Central Cloud services that are expected to be employed for the project lifetime, such as Open Source MANO, OpenWhisk, a standalone, all-in-one OpenStack instance for development purposes and the CI/CD support services (such as Jenkins slave node, container/unikernel repositories, etc.) that will be updated to satisfy the project’s needs. Please refer to the 5G-MEDIA Central cloud resources36 wiki page for further information.

4.2. OTE integrated infrastructure

OTE will provide two separate infrastructures. The first one is based on a VMware platform and the second one on an Open stack platform, also providing two wireless networks based on WiFi and 4G LTE, which offer an end-to-end connectivity. The Open Stack platform will soon be upgraded to one where all the components, controller and nodes will be installed in independent servers.

34 OpenStack API Documentation: https://developer.openstack.org/api-guide/quick-start/ 35 Ceph: https://ceph.com/

36 Central cloud resources (ENG): https://production.eng.it/portal/group/prj_5g_media/wiki/-/wiki/Main/central+cloud+(ENG)


40

4.2.1 VMware cloud

The VMware ESXi cloud operates independently of the other operating systems, which is based on a VMware ESXiv5.5 server supporting the entire deployment. It is a compact architecture supporting the entire VMware infrastructure suite of products including:

• VMware file system, • VMware scheduler and • VMware manager

Its operating system is called VMkernel and all processes run on top of it. VMkernel provides the means for running all processes on the system, including the management apps, agents and all virtual machines. In the following Figure 8, the VMware component architecture is presented.

Figure 8: OTE VMWare component architecture

The virtual machine monitor, VMM, provides the execution environment for a VM.

The console user interface, (CUI), is the low-level configuration and management interface for basic configuration processes.

The Common Information Model, (CIM), module provides the interface for installation of a set of APIs.

4.2.2 OpenStack cloud

4.2.2.1 Current Open Stack Infrastructure

The current Open Stack infrastructure OpenStack is an open source project facilitating us to build our own cloud computing setup. In other words, it creates an Infrastructure as a Service (IaaS) on our own infrastructure. It is a project originally started by NASA and Rackspace for delivering a cloud computing and storage platform. OpenStack lets users deploy virtual machines and other instances that handle different tasks for managing a cloud environment on the fly.

OpenStack software delivers a massively scalable cloud operating system consisting of three major components:


41

1. Compute: open source software designed to provision and manage large networks of virtual machines, creating a redundant and scalable cloud computing platform.

2. Object Storage: open source software for creating redundant, scalable object storage using clusters of standardized servers to store petabytes of accessible data (code-named "Swift").

3. Image Service: provides discovery, registration, and delivery services for virtual disk images (code-named "Glance").

4.2.2.2 Open Stack Lab Topology

OpenStack all-in-one is installed in our lab premises and based on DevStack, which is a specific project used for the basic installation. According to DevStack information, “queens” version has been used and it is installed in an HP ProLiant DL380 Gen9 server with 2 4-cores CPUs, in total 8-cores and 32 GB RAM with Ubuntu 16.04 Xenial OS version. Right now, OpenStack topology for the time being, includes all elements illustrated in Figure 9. As 5G-Media will move on, furthermore instances will be created, such as OpenWhisk, or anything else that 5G-Media requires.

Figure 9: OTE Open Stack all component in one Infrastructure

The connectivity to the internet is done through NAT services with a CISCO ASA network element. The NAT can be configured to be dynamic so that all hosts can reach the internet, or static NAT can be applied to allow also access to particular services from the internal OTE network to be reachable from outside the firewall.


42

Additionally, the 5G-Media partners are able to remotely to the OTE infrastructure via SSL clients through open VPN connections. This allows connectivity to the infrastructure from other developers in the 5G-Media project, (ex. SILO, ENG, IBM, etc.), or from other infrastructure (ex. TID) which is located in Spain. Different credentials are needed to provide to different partners for security but also for management reasons.

The upcoming infrastructure in OTE lab is going to be as follows.

Table 6 - NFVI – PoP nodes

A/A component 1 Cloud controller 2 Compute Node 1 3 Compute Node 2 4 Compute Node 3

The future OTE OpenStack-based infrastructure it will be based on a nova controller together with two-three compute nodes based on neutron agents. The gateway will provide connection to the internet.

Figure 10: Cloud architecture that based on OpenStack

4.2.2.3 Mobile end-to-end system

A mobile system is set-up in the OTE lab that can be used for the delivery of services such as video, etc. It is constituted by two indoor Nokia small base stations of type Flexi zone and a virtual enhanced packet core, (vEPC). The mobility, authentication and roaming of the user’s mobile is performed by the vEPC system. The small base stations are carrier aggregated in two frequencies of 2600 MHz and 1800 MHz and are WiFi capable.


43

Figure 11: OTE Lab mobile infrastructure

The integrated infrastructure in the OTE’s lab is shown in the following Figure 12.

Figure 12: Integrated infrastructure in OTE’s lab

4.3 NCSRD Testbed

The computational resources of the NCSRD testbed are currently:

• One Dell Z230 server used as OpenStack PoP-1 - 1 * Intel(R) Core (TM) i7-4790 CPU @ 3.60GHz - 32GB RAM - 2TB HDD


44

• One HP ProLiant DL380 Gen9 server used as OpenStack PoP-2 - 2 * Intel(R) Xeon(R) CPU E5-2637 v3 @ 3.50GHz - 32GB RAM - 1.2TB HDD

• Two Dell Alienware Desktops with GPU cards used as nodes in the Kubernetes cluster - 1 * Intel(R) Core (TM) i7-8700K CPU @ 3.70GHz - 1 NVIDIA GPU - 32GB RAM - 2TB HDD

and the networking resources are: • A 3Com Baseline Switch 2924-SFP+ • A Cisco 2900 series Integrated Services Router (ISR)

A physical diagram of the integrated testbed in NCSRD is shown in Errore. L'origine riferimento non è stata trovata..

Figure 13: NCSRD Infrastructure

The routing, firewall and access server equipment is based on the Cisco ASA 5510 series. In order to access the NCSRD hosted Infrastructure, AnyConnect VPN or OpenConnect VPN is used. Connection of the Infrastructure to the game server which is located in the premises of CERTH is done with the use of a VPN client of the type “https://vpn.medianetlab.gr".

4.4. Telefonica cloud – OnLife platform

TID Cloud is based on OnLife platform, which is a new architecture for Central Offices (COs) based on edge computing that allows the virtualization of its access network and offers third-


45

party application developers and content providers cloud-computing capabilities at the edge of the network. The integration of OnLife platform with the OSM has been achieved in the framework of 5G-MEDIA, providing a unique advantage of OSM compared to the other MANO solutions. OnLife proposes a design based on NFV, SDN and Cloud Computing. In particular, there were three main principles that guided the design, namely:

Figure 14: Components of CORD

1. Greenfield: No support/integration with legacy systems nor infrastructure,

2. Extreme use of simplification and NFV principles: Making NFVs as simple as possible and only implementing the protocols and pieces of “Central Office Re-architected as a Datacentre” (CORD) that were necessary for the actual services to be provided, but leaving room to increase functionality in the future if needed, and,

3. Open Source Software and Open Compute Project37 (OCP) hardware.

Following the simplification principle, CORD, shown in Figure 14, has been implemented using OpenNebula38, which is a lightweight and flexible solution to enable fast prototyping, service orchestration and management of the “Central Telefónica Procesadora de Datos” (CTpd) infrastructure. In this design, several CORD modules, such as Authentication, Authorization, and Accounting (AAA) or Virtual Tenant Networks (VTN) are not implemented, as its functionality is redundant to other elements to be deployed in CTpd or not required to provide the selected services. The cloud infrastructure provides the flexibility needed to deploy scalable services as well as the management layer for the VMs implementing multiple network functions. In particular, OpenNebula represents a clear advantage in this area compared with other approaches, namely:

37 http://www.opencompute.org/ 38 https://opennebula.org/


46

• Smaller footprint, in terms of hardware requirements, software components and operational costs.

• Simple and extensible architecture, with well-defined interfaces that allow an easy integration of new and innovative scenarios.

• Rich and consistent feature set, to support novel networking services.

Moreover, aiming to a future-proof solution and aligned with the Greenfield principle, CTpd supports only IPv6 and considers IPv4 as a service that interconnects to IPv4 Internet by means of NAT64 at the edge of the data centre or tunnelling the IPv4 customer traffic to that point. Finally, the characteristics of OpenNebula also allow the CTpd to design a network service development framework. This framework will open the CO to third-party applications to explore new business use cases.

ONOS is the selected SDN platform for the deployment of CTpd, as most of CORD functionality relies on applications developed in this platform. More specifically, CTpd focuses on the R-CORD use case, which involves the use of the vOLT and the vRouter applications, already developed for ONOS. Further deployment into the Enterprise and Mobile segments will follow the same approach as the access network becomes one and the same for all segments. Other reasons of choosing ONOS are its simple and dynamic deployment via the Apache Karaf environment, its high-quality documentation and its community support.

4.5. Integration of GPUs in 5G-MEDIA NFVIs

Computing power has traditionally been associated with CPU and RAM availability and so has been, consequently, the demand of resources to cloud providers. In the last years, another kind of processors have gained attention, i.e. Graphical Processing Units (GPUs). Once the last confined to traditional PCs market, now spanning over the area of parallel computing on servers and, therefore, on the cloud.

The reason for the interest on GPUs is the ability to perform high parallel computing based on thousands of cores, while on the other side, multicore CPUs, mainly designed to execute sequential calculation, have only a few dozens of cores and are much less efficient for such kind of computation39.

4.5.1. Integration of GPUs into the cloud environment

Recently, the top cloud providers have started offering GPU based VMs that take advantage of such parallel models and are suitable for the GPU intensive computation (i.e. video rendering, gaming, etc.) and even deep learning computation models. Among the top GPUs suppliers40, NVidia is intensively involved in this transition and has developed a programming model for its GPU called “CUDA”41 to enable developers to build parallel applications, together

39 https://www.forbes.com/sites/janakirammsv/2017/08/07/in-the-era-of-artificial-intelligence-gpus-are-the-new-cpus/ 40 https://wccftech.com/nvidia-amd-discrete-gpu-market-share-report-q3-2017/ 41 https://developer.nvidia.com/cuda-zone


47

with powerful datacenter GPUs boards like the Tesla series42 or the Quadro series boards for desktop and mobile devices43 . According to that, the common ways to access GPUs from VM44 are:

• through Remote API invocation, that allows applications executed within VMs to leverage hardware acceleration with API interception and redirection and achieve high computing performance in a transparent way;

• through Peripheral Component Interconnect (PCI) passthrough 45, that allows a VM to see a host device (such as a GPU board) as if it was physically attached to the guest operating system (see Figure 15) and with near-native performance although with an exclusive usage of such device from the guest operating system;

• through emulated devices (for example with devices supporting SR-IOV46), which introduces a considerable amount of overhead47 especially with GPU devices.

Figure 15: PCI passthrough in virtual environment

Until now, the Remote API invocation approach has been implemented in a number of different ways, including the lately introduced support to CUDA devices (for example vCUDA48

42 http://www.nvidia.com/object/tesla-servers.html 43 https://developer.nvidia.com/cuda-gpus 44 GPU in virtual environment: https://www.isi.edu/sites/default/files/users/jwalters/papers/HPGC_2014.pdf 45 PCI passthrough: https://www.ibm.com/developerworks/library/l-pci-passthrough/index.html 46 SR-IOV: https://en.wikipedia.org/wiki/Single-root_input/output_virtualization 47 overhead in emulated devices: https://www.ibm.com/developerworks/library/l-pci-passthrough/index.html 48 vCUDA: http://ieeexplore.ieee.org/abstract/document/5928326/?reload=true and http://dsc.soic.indiana.edu/presentations/Cloud_2014_GPU.pdf


48

or rCUDA49). However, recent studies show that there are still strong performance issues50, while on the contrary, the PCI passthrough approach (shown in Figure 15) only introduces a small overhead compared to native device usage which represents the best possible use case. Thus, PCI passthrough is a promising candidate solution to be implemented in the scope of 5G-MEDIA project.

4.5.2. GPU provision over containerization technology

In the last years, cloud providers have also started using containerization as a virtualization technology along with the VMs; for this reason, the provisioning of GPUs in the cloud has also evolved embracing such technology (e.g. Docker51) as a more efficient way to manage resources, according to the model known as “Containers-as-a-Service” (CaaS). NVidia has recently released to developers an extension of Docker to support its own GPUs as resources inside containers, with the high-level architecture depicted in Figure 16. With this Docker extension, GPU resources are made available to containers through a “CUDA Toolkit” library, relying on the CUDA driver installed on the host that mounts the NVidia GPU(s). The most important features are basic GPU isolation support52 and the ability to support different version of kernel modules and keep Docker container images portability53, overcoming the dependency on the specific kernel module version due to the necessity of loading user-level NVIDIA GPU driver libraries54.

In Figure 17, the internal architecture of NVidia Docker extension is depicted, as well the actual flow when specific Docker images supporting CUDA are retrieved by the registry55. Each container relies on an NVidia Docker plugin56 that discovers the host driver files and makes the GPU resources available through the NVidia management library57 and even provide GPU monitoring information through REST API interface. The creation of containers is done through the NVidia Docker CLI wrapper, that mounts a host volume containing the hard links to the specific NVidia driver files that are looked up and loaded by the dynamic linker 58 (with ldconfig 49 rCUDA: http://www.rcuda.net/ 50 vCUDA and rCUDA performance compared to native PCI express: https://www.isi.edu/sites/default/files/users/jwalters/papers/HPGC_2014.pdf

51 Docker: https://www.docker.com/

52 Nvidia-docker GPU isolation to assign resources to a single container: https://github.com/NVIDIA/nvidia-docker/wiki/GPU-isolation-%28version-1.0%29

53 Nvidia-docker image portability: https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver-(version-1.0)

54 VM and GPU passthrough setup - comments: https://devblogs.nvidia.com/parallelforall/nvidia-docker-gpu-server-application-deployment-made-easy/ 55 nvidia-docker internal architecture: https://github.com/NVIDIA/nvidia-docker/wiki/NVIDIA-driver-(version-1.0) 56 nvidia-docker-plugin: https://github.com/NVIDIA/nvidia-docker/wiki/nvidia-docker-plugin 57 NVML: https://developer.nvidia.com/nvidia-management-library-nvml 58 dynamic linker: https://en.wikipedia.org/wiki/Dynamic_linker


49

using the ldcache59 cache) in the predefined Docker images supporting CUDA from the Docker Registry. Such CLI wrapper is also able to manage containers on a remote host using network protocols (SSH/HTTP).

Figure 16: NVidia-docker high-level architecture

While the solution provided by NVidia focuses on physical hosts using Docker as virtualization technology, it can also run on a VM-based virtual environment itself, although with some limitations60. For instance, in an approach where PCI passthrough is implemented over OpenStack61, having a VM with NVidia Docker running on top is a simple example 62, but it may potentially become inefficient from the point of view of a cloud provider because of the exclusive usage of the GPU from a single VM. For this reason, the most important container orchestrators have been starting supporting GPUs, and a presentation at the OpenStack Summit in Boston of mid 201763 compared the features available on Kubernetes64,

59 ldconfig and local cache: http://man7.org/linux/man-pages/man8/ldconfig.8.html 60 Docker with GPU on Hyper-V limitations: https://github.com/NVIDIA/nvidia-docker/issues/197 61 PCI passthrough on OpenStack: https://docs.openstack.org/nova/pike/admin/pci-passthrough.html 62 OpenStack GPU passthrough example: https://gist.github.com/claudiok/890ab6dfe76fa45b30081e58038a9215 63 OpenStack and GPU: https://www.OpenStack.org/assets/presentation-media/OpenStack-Boston-Summit-Presentation.pdf 64 Kubernetes: https://kubernetes.io/


50

OpenStack65, Apache Mesos66, and Docker Swarm67, highlighting the actual support to GPU and their limitations, as shown in Figure 18.

Figure 17: NVidia-docker internal architecture

Under this scope, one important feature, particularly in a multi-tenant cloud environment, is the ability to provide GPU isolation, to restrict the usage of a specific GPU to a particular container68. At the time of such comparison (mid 2017), this feature was not completely supported by “nvidia-docker” (different container could get the same GPU69) but only by Kubernetes (version 1.6.x) and Apache Mesos, with the latter using a different containerizer than Docker but using the same approach nvidia-docker adopts to mount the NVidia driver70.

The configuration of such cloud orchestrators and resources together with GPUs on top of different cloud providers has evolved and simplified over time. Tools like Juju from Ubuntu71 allows to model, deploy and manage complex environments through set of scripts (“charms”) publicly available to developers. An example is reported on the official Ubuntu website72 that shows how easy is, with a charm and a few commands, to allocate Kubernetes configured to

65 OpenStack: https://www.openstack.org/ 66 Apache Mesos: http://mesos.apache.org/ 67 Docker Swarm: https://docs.docker.com/engine/swarm/ 68 Nvidia-docker GPU isolation: https://github.com/NVIDIA/nvidia-docker/wiki/GPU-isolation-(version-1.0) 69 nvidia-docker GPU isolation details: slide 24 https://www.OpenStack.org/assets/presentation-media/OpenStack-Boston-Summit-Presentation.pdf 70 GPU support on Apache Mesos: http://mesos.apache.org/documentation/latest/gpu-support/ 71 Ubuntu juju: https://jujucharms.com/ 72 https://insights.ubuntu.com/2017/04/19/how-we-commoditized-gpus-for-kubernetes/


51

provide (NVidia) GPU based containers73 on top of Amazon Web Services74; similar configuration is going to be available for different cloud providers as well75. On the other hand, support to Kubernetes and GPU in OpenStack is still in progress, as shown in the OpenStack Summit in Sydney of November 201776, where has been shown that the support to multi-tenancy requires some specific configuration and effort. The option to deploy Kubernetes 1.6.x. for each tenant with Magnum77 or Heat78 has still some issues79 and possible alternatives are under evaluation80.

Figure 18: Container orchestrators support to GPU (Source: OpenStack Summit 2017

In the context of 5G-MEDIA project, GPUs are used to support the specific tasks described in its use cases81, such as 3D gaming and video rendering. On the other hand, OpenStack is the main NFVI provider as it is supported by different SVPs such as SONATA82 and OSM83,

73 NVidia GPU container support on Kubernetes: https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/ 74 Amazon AWS with GPU: https://aws.amazon.com/ec2/elastic-gpus/ 75 Kubernetes with GPU on Azure: https://medium.com/@wbuchwalter/creating-a-kubernetes-cluster-with-gpu-support-on-azure-for-ml-training-and-predictions-with-a551a19b8859 76 https://www.OpenStack.org/videos/sydney-2017/better-gpu-container-as-a-service-realize-multi-tenancy-with-OpenStack 77 OpenStack Magnum to manage Kubernetes: https://wiki.OpenStack.org/wiki/Magnum 78 OpenStack Heat: https://wiki.OpenStack.org/wiki/Heat 79 limitations of Kubernetes with GPU on OpenStack in multitenant environment: see 4:05 and 4:09 of https://www.OpenStack.org/videos/sydney-2017/better-gpu-container-as-a-service-realize-multi-tenancy-with-OpenStack 80 using Keystone with Kubernetes for GPU multitenancy: see 9:40 of https://www.OpenStack.org/videos/sydney-2017/better-gpu-container-as-a-service-realize-multi-tenancy-with-OpenStack 81 5G-MEDIA use cases: http://www.5gmedia.eu/use-cases/ 82 SONATA: http://www.sonata-nfv.eu/ 83 OSM: https://osm.etsi.org/


52

while Kubernetes is going to be used by OpenWhisk84 to manage clusters of containers. The solution that will be evaluated for 5G-MEDIA project is based on Kubernetes (with a version supporting NVidia GPUs85) running on top of OpenStack and may rely on Magnum86 to supports also other container orchestration engines such as Docker Swarm and Apache Mesos. This may be subject to change over the project time life according to the availability of valuable updates about GPU support from the open source community.

4.6. Multi-PoP (inter-NFVI) network connectivity

The 5G-MEDIA NFVI is composed of different cloud sites geographically distributed and belonging to different organization (i.e. ENG in Italy, TID in Spain and OTE in Greece). Each 5G-MEDIA cloud platform corresponds to a PoP in the NFVI and offers different kind of virtualized resources depending on the Network Service deployment needs. In particular, the 5G-MEDIA NFVI is hierarchically structured with a central cloud platform and edge cloud platforms. The motivation behind this kind of hierarchy is the envisioning of distributed NSs, where VNFs in a VNFFG can be instantiated across the different PoPs depending on their requirements and functionalities. If we take into consideration a typical media service like VoD, reducing latency is a key factor for achieving a high QoE from the end-users. The service provider might want to deploy a cache with the requested content in proximity to the end-user, maintaining the core service functionalities in the central cloud and deploying the new service component in the edge cloud nearest to the end-user. This means that, from the NS management and orchestration point of view, the 5G-MEDIA SVP should have a comprehensive unified view of the different NFVI-PoPs virtualized resources and should be capable of establishing the proper network connectivity between these PoPs in order to guarantee the network communication between VNFs belonging to the same NS.

Depending on the different types of VIM instances deployed in each cloud platform, the implementation of this inter-NFVI PoPs network connectivity functionality may vary. In particular, we distinguish between three different cases:

● Each PoP is under the control of a different type of VIM (e.g. OpenStack, OpenNebula, VMware, etc.).

● The same VIM is used in each PoP (in a following section we are considering the case of OpenStack in a multi-region deployment).

● Hybrid case, where multiple instances of the same VIM coexist with other types of VIM (e.g. multi-region OpenStack plus OpenNebula).

4.5.1 Multi-type VIMs

In the case of an NFVI composed of PoPs managed by different types of VIM (e.g. OpenStack, OpenNebula, VMware, etc.) the inter-NFVI network connectivity should be handled at the NFVO level, in particular integrating the use of a WAN Infrastructure Manager (WIM)

84 OpenWhisk: https://openwhisk.apache.org/ 85 Kubernetes NVidia GPU support: https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/ 86 Magnum: https://wiki.openstack.org/wiki/Magnum


53

component within the NS provisioning workflow, as shown in Figure 19. A WIM is a specialized VIM used for establishing connectivity between endpoints in different NFVI-PoPs.

Figure 19: Interconnecting multiple NFVI-PoPs

Looking at the standards, in order to interconnect the different NFVI-PoPs, the WAN between them should provide Virtual Links, based on the network connectivity services, as requested by the NFVO through the Or-Vi reference point. This implies the introduction of a further level of resources orchestration at the NFVO, where the orchestrator should have an abstracted view of the network topology and interfaces to underlying VIMs and WIMs. In particular, a WIM should implement the following functionalities related with the NFVI connectivity services:

● Path computation according to QoS input parameters ● Connectivity establishing over the physical network ● North Bound APIs to be used by the NFVO ● South Bound APIs / Drivers to SDN Controllers in order to configure the underlying

network The SDN controller invoked by the WIM is responsible for the resource management and for collecting information and statistics about the underlying network (e.g. bandwidth etc.), for such reasons it should implement proper drivers at its South-Bound Interface (SBI) in order to interact properly with the network data-plane.

Different software alternatives already exist for the SDN Controller (e.g. OpenDaylight, ONOS, Floodlight etc.), while the WIM component with its functionalities should be designed from scratch. The integration within an ETSI compliant MANO framework can be handled through the implementation of the Or-Vi reference point as the North-Bound Interface (NBI) of the WIM for the interaction with the NFVO.


54

4.5.2 Multi-region OpenStack

OpenStack is a flexible open source platform providing a well-established framework for managing virtualized resources. However, OpenStack’s networking model, enabled through the Neutron service, was not designed to exchange networking information with external networking entities, in particular other OpenStack instances. In the contrary, the Neutron network model is designed to operate within a single domain and anything outside of that is treated as “external,” preventing the Layer-2 connectivity and requiring the configuration of floating IPs on VMs to be able to reach the “external world”. This subsection will present an architecture alternative for enabling the end-to-end orchestration of distributed NSs across different NFVI-PoPs, where each PoP in the NFVI corresponds to an OpenStack region in a multisite OpenStack deployment.

An OpenStack region can be identified as a regular data-centre, where one OpenStack instance is completely deployed except from the KeyStone component. The KeyStone DB will be centralised or shared between the different regions, enabling authorization and authentication over multiple clouds. What is missing from the NS orchestration point of view is the possibility to have a unified view of the whole NFVI resources offered from the different regions and a way to establish the L2 network connectivity between VNFs within the same NS chain but distributed across different PoPs.

From the networking point of view, a first approach could be the use of Tricircle87, a plugin for networking automation that works across Neutron instances in multi-region deployments, as shown in Figure 20. In order to allow the VNFs inter-connection across regions, Tricircle enables the L2 networking among distributed Neutron instances (e.g. IP address space management, IP allocation and L2 network segment global management). In terms of resource orchestration, Tricircle allows Neutron to work as one cluster in multi-region OpenStack clouds, actuating the orchestration of virtualized networking resources across multiple OpenStack clouds. All VNFs, in an OpenStack tenant domain and provisioned in different clouds, can be interconnected via the global virtualized networking resources.

87 https://wiki.openstack.org/wiki/Tricircle


55

Figure 20: Architecture of Tricircle

The main Tricircle components are the following:

● The Tricircle plugin should be deployed coupled with each Neutron component in the multisite OpenStack deployment. In particular, a kind of hierarchy is needed, deploying a Central Neutron instance with the Tricircle Central Neutron Plugin, which runs under the Central Neutron Server and serves for tenant L2/L3 networking automation. The Tricircle Local Neutron Plugins, deployed in the other PoPs, are responsible for cross Neutron networking automation triggering.

● The Admin API module manages the mappings between OpenStack instances and availability zone, retrieve object uuid routing and exposes API for maintenance.

● The XJob module receives and processes cross-region OpenStack functionalities and other asynchronous jobs from Admin API or Tricircle Central Neutron Plugin.

Concerning the unified view of virtualized distributed resources across different regions, the Kingbird88 project offers a centralized OpenStack service that provides resource operation and management across multiple OpenStack instances in a multi-region OpenStack deployment. In particular, Kingbird provides features like centralized quota management, centralized view for distributed virtual resources, global view for tenant level IP/MAC address space management, synchronisation of ssh keys, images, flavours, security groups, etc. across different PoPs. Kingbird is part of the OPNFV Multisite project that intends to address the use cases related to distributed cloud environments.

In OpenStack, quotas are usually defined per region and are administrated by the OpenStack components, i.e. Neutron, Nova and Cinder. Kingbird, shown in Figure 21 and Figure 22, introduces a new centralised quota management function that allows defining quotas not for

88 https://wiki.openstack.org/wiki/Kingbird


56

region but across the whole NFVI and with a minimal impact on the existing OpenStack services. In particular, the Kingbird quota management function:

● uses existing APIs to dynamically balance quota values ● calculates resource usage upon synchronization ● stores the default/tenant quota limits in the Kingbird DB, providing CRUD operations

for the known quota limits

Figure 21: Kingbird architecture

Figure 22: Kingbird centralized deployment over different OpenStack multi-region PoPs

4.5.3 Multi-region OpenStack in hybrid NFVI

This last case presents the deployment of different types of VIMs for different cloud platforms and the aggregation of some cloud platforms in a multi-region OpenStack deployment as detailed before. This hybrid deployment foresees the use of Tricircle and Kingbird for the OpenStack multi-region installation, as well as the use of a WIM for inter-connecting the OpenStack multi-region deployment, looking as a single VIM and a single PoP, with the other NFVI PoPs managed by different VIMs. Then, also in this case, the inter-NFVI PoPs network connectivity is handled at the NFVO level, where the orchestrator should request the creation of VLs to the WIM through the Or-Vi reference point. The WIM will be responsible for the computation of network paths between the OpenStack multi-region gateway and the other PoPs gateways as well as between the non-OpenStack PoPs. In terms of virtualized resources aggregation across the whole NFVI, this kind of deployment implies also the establishing of a kind of federation in terms of trusts between different administrative domains, since the authentication and authorization systems cannot be shared across the different PoPs in an automatic way and complex configuration might be needed.


57

5. Specifications of VIMs supported by 5G-MEDIA platform 5.1. Integration with OpenNebula

OpenNebula is an Open Source platform oriented of Cloud Computing for datacenters, which allows you to create a virtual infrastructure to build private/public/hybrid Cloud, following the paradigm of Infrastructure as a Service (IaaS). It is free software with Apache2 license. Additionally, OpenNebula orchestrates storage, network, and services. It takes control of the hosts where it deploys multi-tier services such as machines or virtual networks, enabling resources sharing from the own data centre within remote cloud resources.

OpenNebula aims to break through the legacy datacenter infrastructure with specific requirements (as other Cloud Computing platforms), providing flexibility and simple management of the cloud, through the automation and orchestration of simply network operations, storage, virtualization and existing monitoring (Bridges, KVM, VNC, etc.). The main advantage and difference with other similar platforms (e.g. OpenStack) is that OpenNebula is responsible only for the cloud management and has no additional responsibilities. As an example, OpenNebula does not interconnect virtual machines. Instead, it is the SDN controller or the bridge/OVS at a host level that does the work. Another example is the utilization of KVM as a virtualization technology or the use of templates instead of flavours. The difference between templates and flavours is that the last ones are description archives that contain several parameters of the virtual machines (e.g.: Network and execution scripts) while flavours only has CPU, storage and RAM information. The rest of configuration are performed out of the scope of OpenNebula. These are the reasons why OpenNebula is lighter, simpler and easier to install and configure.

In 5G-MEDIA, the integration between OpenNebula and OSM will be developed using a plugin allocated at the OSM’s RO component similar to other cloud platforms, as shown in Figure 23. The objective is that OpenNebula behaves as a VIM additionally to the official VIMs supported Out-of-the-Box by OSM.

Figure 23: Integration between OpenNebula and OSM


58

The plugin will make requests using the XML-RPC API of OpenNebula to fulfil the diverse operations (creation, actualization, etc.) of the necessary virtual services, such as machines or networks. As shown in Figure 24, a similar structure to the existing plugins will be followed, but the content of each method will be modified to fulfil the related request to OpenNebula. At the next step, the received data from OpenNebula will be treated and sent in the proper format for the complete execution of OSM.

Figure 24: Plugin of OpenNebula in RO of OSM

To request XML-RPC from the connector the Python-Oca89 library will be used. This library is in charge of building the HTTP-XML requests and send them to OpenNebula. Below it is shown an example of the Vimconn_opennebula and the use of the Python-Oca library:

89 https://pypi.python.org/pypi/oca


59

Figure 25: Use of python-oca library for XMLRPC OpenNebula Cloud API

5.2. FaaS VIM architecture and Specifications

5.2.1 Background

5G-MEDIA architecture introduces a novel concept of Serverless Virtual Function (SVF). Typically, SVF is an application level Virtual Function (VF) providing a reusable generic or media specific functionality. The complete list of VFs considered in 5G-MEDIA project is included in D2.2. Examples are transcoding, buffering, media clips production, cognitive services, such as speech and image recognition, etc.

In a traditional MANO stack, VFs are provisioned as VMs similarly to the VNFs without any regard to the function life time duration. Nevertheless, an innovation of 5G-MEDIA allows to slash operational costs dramatically, by allowing short lived VFs to be provisioned on demand using Function-as-a-Service (FaaS) programming model. As an example, consider a short-lived media intensive game between two players, where a session lasts only a few minutes. During that session, many VFs should be instantiated: two transcoders for processing the media stream, a buffering function to buffer the last few seconds of the game for on demand clip creation, rendering functions for the two players and for the on-line spectators, and the repeat clip creation function that might be instantiated any time throughout the session to create a clip from the buffered media. FaaS allows to provision all these VFs on demand (typically, in the form of containers) where function provisioning and deprovisioning is handled by a FaaS platform transparently providing for seamless elasticity and scalability. Much of the FaaS benefit comes from the billing model. For each VF/VNF there exists a break even point of time utilization after which FaaS is not justified economically. If, for example, a VF or a VNF should run all the time, FaaS does not offer better cost efficiency than using traditional VMs. To that


60

end, 5G-MEDIA offers FaaS as a virtualization option that can be combined with other forms of virtualization allowing 5G-MEDIA service developers to mix and match the most suitable technologies to attain cost-efficiency.

5.2.2 FaaS VIM Architecture

Figure 26 depicts a reference architecture of integrating FaaS VIM with an ETSI compatible MANO stack. In the 5G-MEDIA reference implementation that uses OSM-R3 some deviations from the standard are unavoidable, because of the gaps that exist between the current OSM implementation and ETSI MANO. In what follows, wherever deviations from the ETSI standard influence the VNFI flows, we specifically mention this and explain 5G-MEDIA specific solution.

5G-MEDIA treats FaaS platforms as virtualization platforms that are managed by their corresponding VIMs, where main reference points Vi-Vnfm, Nf-Vi, and Or-Vi conform to the ETSI MANO specification90. Therefore, communication between NFVO and FaaS VIM VNFM and FaaS VIM is seamless.

Figure 26: Reference Architecture for Integration of FaaS Frameworks with 5G-MEDIA

FaaS is a form of PaaS that takes away the burden of managing underlying virtualized resources, allowing to instantiate VNFs on-demand as processes (e.g., as containers) seamlessly managing their lifecycle. On the south-bound interfaces, FaaS VIM communicates 90 ETSI GS NFV-MAN 001 V1.1.1 (2014-12)


61

with a specific FaaS framework (e.g., Apache OpenWhisk), a specific PaaS (e.g., Kubernetes) and a specific IaaS Cloud Orchestrator (e.g., OpenStack or OpenNebula). In particular, communication between the FaaS VIM and the FaaS framework is required in all VNF lifecycle flows (e.g., instantiation, where instantiation corresponds to invocation of an action). Communication with PaaS might be required in some VNF lifecycle flows, e.g., in one embodiment of this architecture, FaaS VIM communicates with PaaS to provision and deprovision private networks for VNF instances (e.g., as part of instantiation and termination flows respectively). Also, FaaS VIM can communicate with the PaaS framework for the purposes of detailed monitoring. Communication between FaaS VIM and the underlying IaaS Cloud Orchestrator can be required in some flows that are not mandated by the ETSI MANO and this communication is optional. Some cases, where this communication might be required might be related to provisioning of logical networks that can be utilized by the FaaS framework through the PaaS one for providing more separation between the executing actions. Also, this communication might be useful for on demand provisioning/deprovisioning of resources that can be added to PaaS and, therefore, made available to FaaS at time of load peaks/valleys.

Presently there is no standard specified by ETSI for cloud native VNF on-boarding, instantiation and management. Network Service Descriptors (NSD), Virtual Network Function Descriptors (VNFD) and Virtual Deployment Units (VDU) assume hypervisor-based virtualization or bare-metal based infrastructure. In absence of ETSI standard for cloud native PaaS, such as container service and FaaS, we complement the standard specifications with optional elements as explained below. In the future we plan to reach out to ETSI and join forces in standardizing activity with respect to the cloud native PaaS91. A concrete software architecture embodying the presented reference architecture is shown in Figure 27.

91 ETSI GR NFV-IFA 029 v0.50


62

Figure 27: Software architecture for FaaS integration within 5G-MEDIA Platform

Network Service Descriptor (NSD)

Network Services Descriptors are not affected by integrating FaaS with the ETSI MANO framework. In 5G-MEDIA platform architecture we allow mix and match of VNFs within the same VNS, where some VNFs might be deployed as traditional VMs on the virtualized or bare-metal infrastructure managed by their corresponding VIMs and other VNFs might be deployed as containers through the FaaS VIM.

The on-boarding flow of OSM does not change. A VNFD of a FaaS VNF is identical to that of a regular VNF in all mandatory fields. In contrast to the ETSI MANO specified on-boarding flow, OSM-R3 does not validate presence of the VNF image in VIM. In case of FaaS VNFD, the mandatory image reference field contains a fully qualified name of an Open Whisk action implementing this VNF. It is assumed, as for on-boarding of any VNF type that at an arbitrary time, but before the first VNF instantiation, the VNF image should be uploaded to a VIM through which this VNF will be instantiated. Otherwise instantiation will fail. In case of FaaS VNF, uploading of the FaaS VNF image is being done via deployment tools offered by OpenWhisk and extended in this project. In other words, uploading of the FaaS VNF image is done outside of OSM-R3. Figure 28 depicts image upload to FaaS VIM (i.e., to OpenWhisk) process for the general distributed environment for a sample use case corresponding to UC1 of the 5G-MEDIA project, as it is described in D22.


63

Figure 28: FaaS VNF image upload and on-going management

Figure 28 depicts a scenario with a central cloud (at the top) and two edges (at the bottom). This figure focuses on the OpenWhisk assets management and therefore does not show any other components of the platform. OpenWhisk of a central cloud is designated as a “leader”, a single point of truth, with respect to the assets (i.e., FaaS VNFs implemented as Open Whisk Docker actions) of a VNS “Star Ball Game”. The leader OpenWhisk defines a desired state for all edges of the type “game”. The relationships between a leader OpenWhisk and a follower OpenWhisk are logical and can be reversed. However, for maintainability, 5G-MEDIA recommends using central cloud as a central location where FaaS VNFs should be deployed and from which they will be seamlessly propagated by a federation management protocol of OpenWhisk (currently under development). It should be stressed that both leader and follower OpenWhisk instances are independent fully functional OpenWhisk instances.

In Step 1 in Figure 28, the user needs to push a VNF container image to a repository. It can either be a public or a private repository. This should be done for every FaaS VNF in the VNS. In Step 2, the user utilises wskdeploy tool92 to deploy the metadata of all actions pushed to container repository in Step 1. The tool uses a manifest file that describes all the actions which are treated as a managed “project”. After Step 2 completes, the FaaS VNF action metadata is in OpenWhisk of the central cloud and the Docker container image resides in the repository

92 https://github.com/apache/incubator-openwhisk-wskdeploy


64

of the user’s choice and the FaaS VNFs can now be instantiated in the central cloud via OSM’s FaaS VIM plugin that connects to the central cloud’s OpenWhisk.

The assets in the managed project created at the central cloud OpenWhisk will get automatically propagated to the edges that follow that project in Step 4, which occurs as a timer based OpenWhisk action. In case the user wishes to force assets propagation it calls a special OpenWhisk federated management action push_state in Step 4’ specifying the follower OpenWhisk type, game. Configuration of a follower type is part of the OpenWhisk federation management configuration and is performed via a special action create_follower (not shown in the figure) that establishes a logical relationship between the leader and follower OpenWhisk instances with respect to a specific managed project. All federation management actions are provided as system ones to prevent users from accidentally creating actions with the same name in a regular user space.

After the FaaS VNF images and metadata are predeployed as explained, VNF and VNS on-boarding via OSM can be executed. This process does not change because of FaaS integration. More specifically, each FaaS VIM instance is treated as yet another VIM representing its datacenter and a user is given a flexibility to associate a VNF that should be provisioned as serverless with a FaaS VIM of her choice. Figure 29 provides a screenshot of a FaaS VNF on-boarding done via OSM-R3. The reader should notice reference to the “image” of the OpenWhisk action implementing this FaaS VNF. This is a fully qualified action name by which it is accessible in OpenWhisk through the OpenWhisk REST API.

Figure 29: Onboarded FaaS VNFD for a VNF vTranscoder being part of Star Ball Game VNS

After all VNFs of a VNS are onboarded, the user can on-board the VNS itself. Figure 30 shows on-boarded VNS Star Ball Game comprising two FaaS VNFs interconnected by a virtual link.


65

Figure 30: Onboarded VNS Star Ball Game that comprises two vTranscoder FaaS VNFs interconnected by a

virtual link

Next, as shown in Figure 31, the user selects a VNS for instantiation and launches it as shown in Figure 32 and it starts execution as shown in Figure 32. The details about the running VNS can be obtained as shown in Figure 33.

Figure 31: A User selects a VNS to instantiate


66

Figure 32: OSM view of a running FaaS VNS

Figure 33: Status of a running FaaS VNS

Virtual Network Function Descriptor (VNFD)

VNFD contains all information necessary to deploy a VNF. Since the assumed underlying technology is hypervisor based virtualization, vnfd:vdu information elements related to hypervisors include hypervisor_parameter elements with at least one parameter being mandatory. There is one special case, where there is no virtualization and a VNF should be


67

provisioned on a non-virtualized system, but still managed through Virtualized Infrastructure Manager. To allow for this use case, ETSI standard93 allows to specify a reserved label “BARE” as a hypervisor type parameter (see Table 7). This constitutes some minor abuse of notation but allows for a practical and efficient workaround. In the same spirit, we propose to use vnfd:vdu information elements related to hypervisors to describe information pertinent to FaaS based deployment of a VNF.

OSM-R3 ignores vnfd:vdu related to hypervisors. However, we believe that in the future releases this might change and OSM will validate vfnfd:vdu hypervisor related information. To this end in a forward-looking manner, we propose “FaaS” to be used to describe a hypervisor type and also add FaaS related parameters, such as specific FaaS framework type (e.g., OpenWhisk), OpenWhisk version, required PaaS service (e.g., Kubernetes), specific version of Kubernetes, etc. The resulting vdu will be a comprehensive description of the underlying FaaS technology required for this VNFD. This allows easy integration with any FaaS and PaaS and facilitates validation during the on-boarding and instantiation flows. It should be stressed that this is the only mandatory extension in VNFD that is required to seamlessly integrate FaaS with the rest of the MANO stack.

Table 7 - ETSI Standard: vnfd:vdu information related to hypervisors

Identifier Type Cardinality Description

hypervisor_parameter Leaf 1…N There are a number of hypervisor related parameters that can have a significant impact on the deployment and performance of the VDU. These include: • Hypervisor type (see note 1) • Hypervisor version as a VDU may be validated with a particular version. • Hypervisor Address Translation support parameters including:

o Second Level Address Translation (see note 2). o Second Level Address Translation with Large page support. o Second Level Address Translation for I/O. o Second Level Address Translation for I/O with Large page support. Where "Large" is considered to be 1 GB or greater. o Support for interrupt remapping, i.e. supporting the IOMMU in the hypervisor. o Support of data processing acceleration libraries in the hypervisor, i.e. for acceleration libraries which require hypervisor support for high performance

NOTE 1: "BARE" could be included in this list if the network function component was to be deployed on a non-virtualised system, but still managed via the Virtualised Infrastructure Manager. NOTE 2: This needs to be included as the platform may support this feature, but the hypervisor may not.

93 ETSI GS NFV-MAN 001 V1.1.1 (2014-12), p.49


68

Conclusions In this deliverable, we have presented the initial design and main technical specifications of the 5G-MEDIA Service Virtualization Platform. The SVP constitutes the central component of the 5G-MEDIA architecture, being responsible for the service orchestration and the intelligent deployment of the media VNFs/NSs on the physical resources. In 5G-MEDIA, the development of the SVP will rely on the Open Source MANO (OSM) release 3.0 and will integrate, extend and adapt the results from 5GPPP Phase 1 projects (CogNET, SELFNET, SONATA). The resulting platform will be instantiated, and proof-of-concept validated considering three NFVIs with different characteristics (e.g., different virtualization technologies used) provided by the project partners ENG, OTE, TID.

The main contributions of this deliverable lie in the technical specifications design for each 5G-MEDIA SVP component (Service Orchestrator, Media Service MAPE, Resource Orchestrator), in light of the existing technologies and prototypes to be used during development as well as in the presentation of the target NFVIs, their characteristics and peculiarities to be considered during their integration in the 5G-MEDIA platform. More specifically, in the context of the service orchestrator, 5G-MEDIA will support the following characteristics: i) flexible configuration of the VNFs/NSs through their descriptors to allow intelligent instantiation supported by Media Service MAPE ii) deployment of VNFs/NSs in distributed NFVIs; iii) continuous monitoring of deployed VNFs/NSs and capability for upscaling/downscaling depending on the application needs; iv) updates of VNFs/NSs instantiation based on Media Service MAPE outputs. In the case of Media Service MAPE, the main specifications identified are: i) selected monitoring tools for different virtualization technologies (OpenStack, VMWare, OpenNebula); ii) specification for real-time data communication between monitoring and CNO module; iii) specification of the scenarios supported by the machine learning algorithms including in the CNO, such as Mobile User Throughput Prediction, Anomaly Detection etc. iv) specification for the policy engine, part of the CNO, which will support intelligent VNF/NS placement, VNFFG optimization etc. For the Resource Orchestrator, the main achievements foreseen in 5G-MEDIA are: i) the implementation of the OpenNebula VIM to allow OnLife integration; ii) the development of FaaS VIM to enable the exploitation of the FaaS concepts in 5G. Finally, the three target NFVIs are presented illustrating the differences in terms of virtualization technologies (OpenStack, OpenNebula, VMWare), topologies and specific requirements (e.g., need for GPUs integration, need for Multi-PoP connectivity).

Overall, the deliverable provides an initial point of reference for the development teams to start the implementation of the integration and extensions of the baseline technologies. This initial design and specification is expected to evolve during the implementation phase and it will be refined in the deliverables “D3.2 Specification of the G-MEDIA Serverless Computing Framework” and “D3.3 Specification of the 5G-MEDIA QoS Control and Management Tools”.


69

Annex I – Installation steps of OSM v3.0 The flow of the commands that we used are:

> ssh -i <pem> ubuntu@ipv4 # Access the OSM VM

$ sudo apt-get update

$ sudo apt-get install -y lxd

$ newgrp lxd # add lxd user/group

$ sudo lxd init # Configure LXD - select the default options in the dialog

# Enable the IPv4 and NAT

# Disable the IPv6

# LXD creates a bridge named lxdbr0.

$ lxc list # This will drive initialization of lxdbr0

$ ip address show # Check the interfaces

$ ip address show ens3 # Inspect the default interface and check the MTU (i.e. 1450)

$ ip address show lxdbr0 # Inspect the bridge

# If MTU of ens3 and lxdbr0 differs, we use the ens3 MTU

$ sudo lxc profile device set default eth0 mtu 1450 # Use the appropriate MTU value

# Check if LXD has installed properly

$ lxc launch ubuntu:16.04 test # Create a container based on Ubuntu 16.04 with name 'test'

$ lxc exec test bash # Access the container

$ root@test:~# apt-get update # Run command 'apt-get update' from inside the container

$ root@test:~# exit # Exit from the container

$ lxc stop test # Stop the container

$ lxc delete test # Delete the container

# Download the OSM release 3

$ cd $HOME # /home/ubuntu

$ wget https://osm-download.etsi.org/ftp/osm-3.0-three/install_osm.sh

$ chmod +x install_osm.sh


70

# Install it through the bash script

$ ./install_osm.sh

# After approx half an hour, the installation will be completed...


71

Annex II – Data model for VNFDs in OSM v3.0

OSM key Name OSM type

High level Description

Group of fields Description of the fields

id String Identifier for the NSD

name String NSD name

short-name String NSD short name to use as a label in the UI

vendor String Provider of the NSD.

logo String File path of the vendor-specific logo

description String Description of the NSD

version String Version of the NSD

connection-point List List of the connection points of the NS.

● Name ● type The Name and the type of

the NS connection-point. Only value VPORT (virtual port) is supported.

vLd List List of the Virtual Link Descriptors of the NS.

● ID ● name ● short-name ● vendor

ID is the [key] identifier for the VLD. Vendor holds the name of the provider of the VLD.

constituent-vnfd List List of the VNF Descriptors that are part of this Network Service.

● Member-vnfd-index ● vnfd-id-ref ● start-by-default

Holds the unique id of the VNFD.

Holds the path of the VNFD [id].

States if the VNFD is started as part of NS instantiation.

scaling-group-descriptor

List List of scaling groups of the NS. The scaling groups consist of one or more VNFs. For each

● Name Name of the scaling-group .


72




scaling group a different scaling action may be applied.

● scaling-policy Contains Scale-in/out

criteria for this network service.

● vnfd-member (ui32) Contains the list of the

member VNF index and the number of each VNF's instances when a scaling action targets this group.

● min-instance-count ● max-instance-count

Minimum and maximum instances of that are allowed.

● scaling-config-action Contains the scaling trigger type (pre/post - scale in/out) and a reference to the NSC primitive name.

placement-groups

List List of placement groups. Each placement group defines the compute resource placement strategy in cloud environment.

● Name ● requirement ● strategy ● member-vnfd

Contains the resource placement strategy (COLLOCATION or ISOLATION) for a particular set of NS VNFDs.

Ip-profiles-list List List of IP profiles that describe the IP characteristics for the Virtual Link.

● Name ● Description ● ip-profile-params

Contains IP parameters for each profile. (IPv, subnet, gateway, DNS list,DHCP,subnet-prefix-pool/ VIM-specific reference)

vnf-dependency List List of VNF dependencies.

● Vnf-source-ref ● vnf-depends-on-ref Describes which VNF

depends on the source VNF.


73




vnffgd List List of VNF forwarding graph descriptor.

● Id ● name ● short-name ● vendor ● description ● version ● RSP ● classifier

Contains fields that uniquely describe each FG. Each VNFFG model consists of a list of rendered service path (RSP) and a list of classifier components. RSP is an ordered list of references to SF. The SF reference exists in the form of references to connection-points of the constituent-VNFDs. Classifier defines a list of rules for the VNFFGD.

monitoring-param

List List of network parameters for the NS.

● Id ● name ● monitoring-param-ui-data ● monitoring-param-value ● aggregation-type ● vnfd-monitoring-param,

Contains fields that refer to parameters for:

● UI-data (eg. Histogram, bar, units/Mbps)

● values (eg. INT,STRING)

● aggregation-types (eg. AVG,SUM)

● Reference to VNFD or VNFD monitoring parameter

input-parameter-xpath

List List of xpath to parameters inside the NSD that can be customized during instantiation.

● Xpath ● Label ● default-value

Xpath that specifies the element in a descriptor, along with a descriptive string (label) and a default value for this input parameter.

parameter-pool List Pool of parameter values that must be pulled from during configuration.

● Name ● range Name of the conf value

pool and a Range of values from which to populate the pool. (Start/End values)

service-primitive List Network service level configuration primitives.

● Name, ● parameter ● parameter-group ● vnf-primitive-group ● user-defined-script

Holds the name of the service primitrive, a grouping of parameters that are logically grouped in UI, a List of service parameters grouped by the


74




VNF and a user defined script.

Parameters of a Service-Primitive include:

● name ● data-type

(string,int,boolean)

● mandatory (boolean)

● def_value

initial-config-primitive

List Set of configuration primitives to be executed when the network service comes up.

● seq ● name ● user-defined-script ● parameter

Holds the sequence number (seq) for the configuration primitive, its name, a user-defined script and a list of parameters with the subfields (name, value).

terminate-config-primitive

List Set of configuration primitives to be executed before during termination of the network service.

● seq ● name ● user-defined-script ● parameter

cloud-config List Configures the list of users and public keys to be injected as part of network service instantiation.

● key-pair (name,key) ● user (name, user-info, key-

pair)


75

Annex III – Data model for NSDs in OSM v3.0

OSM key Name

OSM type

High level Description Group of fields Description of the fields

id String Identifier for the VNFD

name String VNFD name

short-name String VNFD short name to use as a label in the UI

vendor String Provider of the VNFD.

logo String File path of the vendor-specific logo

description String Description of the VNFD

version String Version of the VNFD

vnf-configuration

container Information about the VNF configuration for the management interface.

● config-method Defines the

configuration method for the VNF

● script (BASH, EXPECT)

● Juju charm to use with the VNF

● config-access IP, username and password to be used to configure this VNF

● config-attributes

● config-priority ● config-delay

● service-primitive Holds the name for

the config primitive, a list of parameters and a user defined script.

● initial-config-primitive Initial set of

configuration primitives.


76

OSM key Name

OSM type


● config-template Configuration

template for each VNF.

mgmt-interface

container Interface over which the VNF is managed.

● endpoint type (IP, vdu-id,cp)

● port ● dashboard-

params (path, https, port)

internal-vld List List of Internal Virtual Link Descriptors (VLD).

● ID ● name ● short-name ● vendor ● description ● version

Unique description of the internal virtual link

● type ELAN - multipoint

service that connects a set of VDUs

● root-bandwidth

For ELAN this is the aggregate bandwidth

● leaf-bandwidth

For ELAN this is the bandwidth of branches

● internal-connection-point

List of internal points in this VLD represented as id-refs

● virtual-connection-points

List of (available) virtual-connection points associated with this virtual Link.

● name, id, short-name

● type (VPORT)


77

OSM key Name

OSM type


● port-security-enabled (if true RO passes the value to the VIM to filter traffic.)

● static-ip-address

● associated-cps (list of id-refs of cp associated with vcp )

● provider-network

● physical-network

● overlay-type (identifies the type of the overlay network - LOCAL, FLAT, VLAN, VXLAN, GRE)

● segmentation-id

init-params ● vim-network-name

● ip-profile-ref

Ip-profiles List List of IP profiles that describe the IP characteristics for the

Virtual Link.

● Name ● description ● ip-profile-

params

Contains IP parameters for each profile. (IPv, subnet, gateway, DNS list,DHCP,subnet-prefix-pool/ VIM-specific reference)

connection-point

List List for the external connection points

● name ● id ● short-name

Unique description of the cp.

● type VPORT

● port-security-enabled When set to True,

the resource orchestrator passes the value to the


78

OSM key Name

OSM type


VIM when the connection-point is created to filter traffic. (OpenStack VIM only)

● static-ip-address IPv4 or IPv6

vdu List List of virtual deployment units (VDUs).

● id ● name ● description ● count ● mgmt-vpci ● vm-flavor ● guest-epa ● vswitch-epa ● hypervision-

epa ● host-epa ● alarm ● image-

properties ● cloud-input ● supplemental-

boot-data ● internal-

connection-point

● internal-interface

● external-interface

● volumes

VDUs are virtual machines that host the network function, such as:

● Virtual machine specification

● Computation properties (RAM size, disk size, memory page size, number of CPUs

● number of cores per CPU, number of threads per core)

● Storage requirements

● Initiation and termination scripts

● High availability redundancy model

● Scale out/scale in limits

vdu-dependency

List List of VDU dependencies, from which the orchestrator determines the order of startup for VDUs.

● vdu-source-ref

● vdu-depends-on-ref

A reference to the VDU on which the source VDU depends on.


79

OSM key Name

OSM type


service-function-chain

enum Type of node in the service function chaining (SFC) architecture: UNAWARE (default), CLASSIFIER, SF, SFF

service-function-type

String Type of service function. This field is temporarily set to string data type for ease of use.

monitoring-param

List List of network parameters for the VNF.

● http-endpoint ● monitoring-

param ● monitoring-

param-ui-data ● monitoring-

param-value

Contains fields that refer to parameters for:

● http-endpoint (list of http endpoints to be used by monitoring params)

● monitoring-param/ui-data (includes parameters for JSON queries, http-endpoint reference and UI monitoring parameters as described for NSD)

placement-groups

List List of placement groups at VNF level. The group construct defines the compute resource placement strategy in cloud environment

● name ● requirement ● strategy ● member-

VDUs

Contains the resource placement strategy (COLLOCATION or ISOLATION) for a particular set of NSs/VNFDs. Also contains a list of VDUs that area part of this placement-group.

Programmable edge-to-cloud virtualization fabric for the ... · Project co-funded by the European...

Documents

Transcript of Programmable edge-to-cloud virtualization fabric for the ... · Project co-funded by the European...