Unicorn Libraries, IDE Plugin, Container Packaging and...

D2.1 Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset Early Release

1

Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset Early Release Deliverable D2.1

Editors Athanasios Tryfonos

Demetris Trihinas Reviewers Sotiris Koussouris (SUITE5)

John Samuelsson, Erik Robertson (REDIKOD) Date 30 March 2018 Classification Public


2

Contributing Author # Version History

Name Partner Description

Demetris Trihinas UCY 1 Table of Contents (ToC) and partner contribution assignment.

Athanasios Tryfonos UCY 2 Introduction of Section 5 and subsection 5.2

Zacharias Georgiou UCY 3 Finalized state-of-the-art and key technologies challenges and updates section.

George Pallis UCY 4 Initial content for subsection 5.1 has been added.

Julia Vuong CAS 5 Finalized Section 1

Manos Papoutsakis FORTH 6 Monitoring and Elasticity, Perimeter Security and Vulnerability Assessment models are added

Giannis Ledakis UBITECH 7 Deliverable ready for review.

8 Addressed comments from reviewers and compiled final version ready for submission


3

Contents

CONTENTS 3

1 INTRODUCTION 9

1.1 Document Purpose and Scope 10

1.2 Document Relationship with other Project Work Packages 11

1.3 Document Structure 11

2 STATE OF THE ART AND KEY TECHNOLOGY AXES CHALLENGES AND UPDATES 12

2.1 OASIS TOSCA Specification 12

2.2 Docker Container Technology Stack 13

2.3 Service Mesh and Container Design Patterns 15

3 UNICORN DESIGN LIBRARIES 18

3.1 Overview 18

3.2 Monitoring and Elasticity 18 3.2.1 Design Library Overview 18 3.2.2 Design Library Model Description 20

3.3 Perimeter Security and Risk & Vulnerability Assessment 23 3.3.1 Design Library Overview 23 3.3.2 Design Library Model Description 26

3.4 Privacy and Authorization 28 3.4.1 Design Library Overview 28 3.4.2 Design Library Model Description 30

4 DESIGN LIBRARIES IMPLEMENTATION 32

4.1 Design Libraries Implementation Aspects 32

4.2 Unicorn Service Graph Representation 33

4.3 Unicorn Compose File 36 4.3.1 Unicorn Compose Descriptor Interpreter 37

4.4 Unicorn Enabled Container Packaging 38

4.5 Unicorn Design Libraries Features 40 4.5.1 Monitoring and Elasticity Design Libraries Features 40 4.5.2 Perimeter Security Design Library Implementation 41 4.5.3 Risk and Vulnerability Assessment Implementation 41 4.5.4 Privacy and Authorization Implementation 43

5 UNICORN DASHBOARD 43

5.1 Unicorn Management Perspective 44 5.1.1 Entities and Entity Relationship in Unicorn Dashboard 44 5.1.2 Technical Approach 46 5.1.3 Provided Functionalities 47 5.1.4 Exposed APIs 51

5.2 Unicorn Development Perspective 56

6 CONCLUSION 60


4

7 REFERENCES 61

8 ANNEX 62

8.1 Unicorn Enhanced Docker-Compose Model Specification 62 8.1.1 The format and the specification of the Unicorn Docker-Compose model 62 8.1.2 Unicorn Compose File Extended Example 70


5

List of Figures Figure 1 Unicorn Reference Architecture 10 Figure 2 Structural Elements of a Service Template and their relationships 12 Figure 3 Abstract view of the Docker Engine 13 Figure 4 Docker Swarm can be swapped out with different container orchestration technologies such as Kubernetes or Mesos 14 Figure 5 Docker-Compose that consists of 3 services; i) redis service which is bound onto frontend network, ii) db service that mounts a volume on the host machine called db-data and bound on the backend network and iii) a vote service bound on frontend network 14 Figure 6 x-custom special field definition. This field is not in the semantics of docker-compose language specification, however with its definition it is being ignored or handled in a different manner by docker-compose. 15 Figure 7 Example of a sidecar container that syncs the file system with a git repository 15 Figure 8 Example illustrating the Ambassador design pattern 16 Figure 9 Example illustrating the Adapter design pattern 16 Figure 10 Istio Architecture 17 Figure 11: Elasticity Policy Example 19 Figure 12 The security model conforming to the abstract syntax of the security meta-model 25 Figure 13 ABAC Model for Unicorn 30 Figure 14 Maven Import of Unicorn Monitoring Library 32 Figure 15 Timeit annotation that calculates the time needed for a method to be executed 32 Figure 16 Example of Unicorn Monitoring Annotation 33 Figure 17 Simple Service Graph with properties 34 Figure 18 Representation of Service Graph as Tosca YAML 35 Figure 19 The Unicorn Descriptor Interpreter 38 Figure 20 Example Dockerfile that builds a Unicorn parent image based on Ubuntu 16.04 40 Figure 21 The Dashboard layer divided into two perspectives. 44 Figure 22. Entities in Unicorn Dashboard. 45 Figure 23. Entity Relationship in Unicorn Dashboard. 46 Figure 24 Eclipse Che Architecture 57 Figure 25 Unicorn Che IDE Plugin High-Level Architecture 57 Figure 26 Unicorn Stack Configuration for Che 59

List of Tables Table 1 Metric EBNF Definition 20 Table 2 Monitoring Agent EBNF Definition 20 Table 3 Insight EBNF Definition 21 Table 4 Elasticity Policy EBNF Definition 22 Table 5 Elasticity Trigger EBNF Definition 22 Table 6 Elasticity Action EBNF Definition 23 Table 7 Perimeter Security and Risk & Vulnerability Assessment EBNF 26 Table 8 EBNF Definition of the ABAC model used in Unicorn 30 Table 9 Docker container requirements for Unicorn enabled microservices 39 Table 10 Monitoring and Elasticity Table of Features 40 Table 11 Perimeter Security Table of Features 41 Table 12 Risk and Vulnerability Assessment Table of Features 41 Table 13 Privacy and Authorization Table of Features 43 Table 14 User Registration Functionality 47 Table 15 Login/Logout Functionality 47 Table 16 Delete User Functionality 48


6

Table 17 Update User Profile Functionality 48 Table 18 Change Authorization Details Functionality 48 Table 19 Register a new Usergroup Functionality 48 Table 20 Update an existing usergroup Functionality 48 Table 21 Start a Unicorn-compliant app Functionality 49 Table 22 Stop an application Functionality 49 Table 23 Delete application Functionality 49 Table 24 Deploy Unicorn Compliant Application Functionality 49 Table 25 Get Monitoring Data Functionality 50 Table 26 Update Runtime Policies 50 Table 27 Update Service Graph Functionality 50 Table 28 Cloud Offering Marketplace Functionality 50 Table 29 Generate Service Graph Functionality 51 Table 30 Update Service Graph Functionality 51 Table 31 Launch Unicorn IDE Plugin 51 Table 32 Update Usergroup API 52 Table 33 Delete Usergroup by ID API 52 Table 34 Update User API 52 Table 35 Delete User API 53 Table 36 Get all Users API 53 Table 37 Get User by ID API 53 Table 38 Enable Registered user API 54 Table 39 Disable User API 54 Table 40 Create new Usergrup 54 Table 41 Register user API 55 Table 42 Login API 55 Table 43 Update Credentials API 56


7

Executive Summary The purpose of this deliverable is to report the progress made for the ongoing work regarding workpackage 2 and more specifically to document the early release of the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset. The deliverable begins by presenting an updated state-of-the-art and key technology axes section. We identify and analyze technologies that are introduced for the first time within Unicorn and identify challenges and potential obstacles that current technologies may impose during the implementation and integration phases. More specifically, we introduce the concepts of sidecar architecture which is a multi-container architectural pattern that extends the functionality of a main container with additional containers, without performing any changes to its structure and functionality. Also, we present leading technologies that adhere to the sidecar architecture, such as Google’s Istio and Envoy Proxy, which will be used in the implementation of Unicorn. We elaborate on the concept of the service graph by adopting the OASIS Tosca specification, since the interrelation between nodes and edges of a graph can be mapped to structural components of the Tosca Specification. The final focus on this first section of the deliverable is on Docker and its technology stack and various tools that play an important role in Unicorn. Unicorn Design Libraries, assist developers to design cloud applications as Unicorn-enabled micro-services, with the minimum code intrusion that is required to enable portable security enforcement mechanisms, monitoring and elastic scaling, and to ensure data privacy constraints. To this end, Unicorn offers four design libraries namely i) Monitoring and Elasticity, ii) Perimeter Security, iii) Risk and Vulnerability Assessment and iv) Privacy and Authorization design libraries; their meta-models and language are presented in EBNF format later on Section 3. It should be noted that due to similarities and interdependencies of the concepts of the aforementioned design libraries, some are presented and represented as a single library, i.e. Monitoring and Elasticity and Perimeter Security with Risk and Vulnerability assessment. Minimum code intrusion in the development of Unicorn-enabled microservices is achieved by annotating specific bits of code, methods or variables with decorators that provide additional functionality, without being a necessity for developer to have deep knowledge of the annotating concepts. Even though deep knowledge of the concepts is not required, developers should at least provide initial policies and feature configuration that match their requirements for the developed microservice. For this purpose, Section 4 introduces a table with the features and policies that each design library exposes as functionality and at which stage of the continuous development and integration pipeline this feature applies to (it is either at design-time, using annotations or at run-time through configurations at the dashboard). Furthermore, this Section presents the runtime environment at the level of the container and how user-developed applications are packaged within Unicorn Docker containers. To this end, Unicorn prepares pre-made Dockerfiles that describe images that have pre-bundled all the Unicorn execution agents such as monitoring and security within containers. In addition to these, within the context of Unicorn, the Docker-Compose file is extended with a semantic language that expresses added Unicorn functionality as described by the Unicorn enhanced service graph. Finally, this deliverable presents the logical separation and the structure of the development and management perspectives within the Unicorn Dashboard. The development perspective is built on top of the popular Cloud IDE Eclipse Che as an extension plugin, while the management perspective is built as a separate web front-end application based on popular web technologies such as Javascript and AngularJS. Through IDE at the development perspective, developers are able to use the Unicorn Design Libraries to annotate their source code and test their microservices at Unicorn enabled execution environments that simulate production environments. At the management perspective, cloud application administrators have the ability to create and edit the service graphs of the deployed applications, apply and edit run-time policies, manage and register users and monitor the behavior of deployed applications.


8

Table of Abbreviations

ABAC Attribute-based access control

ACM Access Control Mechanisms

AOP Aspects Oriented Programming

API Application programming interface

CLI Command Line Interface

EBNF Extended Backus-Naur form

EMF Eclipse Modeling Framework

IDE Integrated Development Environment

IDS Intrusion Detection System

IPS Intrusion Prevention System

JDK Java Development Kit

JSON JavaScript Object Notation

JSR Java Specification Requests

MVC Model View Controller

REST Representational State Transfer

TOSCA Topology and Orchestration Specification for Cloud Applications

UI User Interface

WP Work package

XACML eXtensible Access Control Markup Language

XML Extensible Markup Language

YAML YAML Ain't Markup Language


9

1 Introduction Unicorn project offers a novel and unified framework that facilitates DevOps teams to design, deploy and manage secure and elastic-by-design cloud applications which follow the micro-service architectural paradigm and which can be deployed over multi-cloud containerized execution environments. Deliverable 2.1, henceforth simply referred to as D2.1 provides a comprehensive documentation report of the early release of the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset. Figure 1 depicts the latest version of the Unicorn Reference Architecture, with components relative to this deliverable being highlighted. At the IDE layer, Unicorn can be separated into two distinct perspectives, the Development Perspective and the Management Perspective, each focusing on specific user roles and offering specific capabilities to the related users. The Development Perspective is built on top of the popular cloud-based IDE Eclipse Che [1]. It offers a collaborative development environment with configurable dockerized runtime workspaces to facilitate the deployment and testing of the injected Unicorn features. Bundled within the configurable runtime workspace, the Unicorn Design Libraries assist developers to write Unicorn enabled apps while keeping the code intrusion to the minimum with the usage of annotations. The Management Perspective provides Cloud Application Administrators with the ability to view in an intuitive graphical manner real-time monitoring data capturing the application behavior and performance of the underlying platform, assessment of application elastic behavior and potential security incidents. The Management Perspective supports also the composition and editing of application service graphs, which represent the topology and interconnectivity of deployed applications. It also makes possible the application and editing of runtime application policies and constraints either on nodes or edges of the service graph. It should be noted that the Management Perspective is developed separately from Che, although there is the possibility through a menu link to launch the Development Perspective. The other critical component being reported in this deliverable is the set of the four Unicorn Design Libraries. As already mentioned in previous Unicorn Deliverables D1.1 [2] and D1.2 [3], Unicorn will provide the concepts of Monitoring and Elasticity, Perimeter Security, Privacy and Risk and Vulnerability Assessment through annotations that can apply on different layers at the source code level. The whole annotation and annotation interpretation process are transparent to the developer and do not require additional effort or specialized know-how for these topics. As already mentioned, microservice [4] compositions are represented as service graphs. At the dashboard layer the service graph is visually depicted as nodes and edges. What is more interesting and useful is the formal representation of the service graph at the components closer to the Unicorn Platform and the Cloud Execution Environment. At those levels, a service graph is translated to a Docker-Compose [5] YAML file. Docker-compose is a tool for defining and running complex applications consisted by multiple Docker containers [6]. At Unicorn, in order to provide the enhanced Unicorn functionality at the deployed Docker containers, there needs to be a way for policies defined through the Design Libraries to be reflected at the Docker-Compose service graph description. To this end, Unicorn extends the Docker-Compose syntax with the ability to define run-time policies and the behavior for Unicorn enabled microservices. Finally, this deliverable presents the output of the container packaging which is a Unicorn Docker container. A Unicorn Docker container has pre-bundled specific execution environment agents and processes such as the Monitoring Agents and the Monitoring Probes which are responsible for the collection and dissemination of application and container measurements, the Snort IDS [7] that protect the user deployed container from insecure inbound and outgoing traffic and the Envoy Proxy [8] which is responsible for traffic management and communication between interconnected microservices.


10

Figure 1 Unicorn Reference Architecture

1.1 Document Purpose and Scope The purpose of this deliverable is to provide a comprehensive overview and documentation report of the early release of the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset. D2.1, which is under the umbrella of Work Package 2, consists of the following components: i) The Unicorn Dashboard with the Development and Management Perspectives; ii) The Monitoring & Elasticity, Perimeter Security, Privacy and Risk & Vulnerability Assessment Design Libraries, and iii) the container packaging and deployment phase. D2.1 documents the Dashboard functionalities and the workflows for the Management and the configuration and setup of the Development Perspectives. For each one of the four design libraries mentioned above, D2.1 presents an analytical model expressed in a formal language and a list of features and policies that are exposed to developers either via annotations or at runtime through the management perspective of the Dashboard. Finally, D2.1 documents what happens at the container packaging and deployment layer. More specifically, it


11

describes the base Unicorn Docker image which contains Unicorn execution environment and a properly configured Envoy proxy to realize the service mesh architecture.

1.2 Document Relationship with other Project Work Packages This deliverable is built on the foundation of D1.2, which provides a concrete documentation of the current version of the reference architecture and key technologies supported by Unicorn and provides an initial description of the components comprising the Unicorn framework. To this end, D2.1 extends the Unicorn documentation by providing a comprehensive report for the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset. What is more, D2.1 serves as a guide for D2.2, the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset Final Release, which will assess the accomplishment of the requirements, features and toolsets introduced in this deliverable and will provide the final documentation of the Unicorn Libraries, IDE Plugin, Container Packaging and Deployment Toolset.

1.3 Document Structure The rest of this deliverable is structured as follows: Section 2 Performs an update regarding key technological axes, introduces new technologies and focuses on challenges that the current state of the art technologies is facing. More specifically, it introduces the connection between the OASIS TOSCA specification and the service graph. It elaborates on the Docker family technology stack and service mesh and sidecar architecture. Section 3 Provides a description of the monitoring & elasticity, security, risk & vulnerability and privacy design libraries and present their models in a formal language. Section 4 Describes the implementation aspects of the design libraries. What is packaged within a Unicorn enhanced Docker container and how a Unicorn enhanced service graph with policies maps to the Unicorn extended Docker-compose file. Finally, this section is concluded by presenting a list of features and tables that each design library offers to developers. Section 5 Describes the Unicorn Dashboard and implementation aspects of the two main perspectives of the Dashboard which are: i) The Management Perspective and ii) The Development Perspective (Unicorn Cloud IDE Plugin). Section 6 Concludes the deliverable.


12

2 State of the Art and Key Technology Axes Challenges and Updates This section acts as a complementary and additional material to the state-of-the-art that was extensively presented in D1.2 [3]. In this section we are introducing potentially new terminology such as the Service Mesh and the Sidecar pattern, analyzing in more depth the Docker technology stack and provide a more comprehensive view on how OASIS Tosca [9] can be used to describe a service graph. Furthermore, some key challenges will also be presented, when and if possible, that are related with these state-of-the-art technologies within Unicorn context.

2.1 OASIS TOSCA Specification In D1.2 OASIS Tosca [9] was introduced as a specification that describes application portability, interoperability and management features of modern cloud applications. The standard is used to describe both the topology of cloud-based Web services, consisting of their components, relationships, and the processes that manage them, and the orchestration of such services—that is their complex behavior in relation to other described services. The Tosca specification introduces a language expressed either in XML or YAML format for describing service templates by means of Topology Templates and Plans in order to define the structure of a service. The two basic building blocks of a Topology Template are nodes and relationships that define the topology model of a service as a (not necessarily connected) directed graph which is illustrated in Figure 2. Nodes in the Topology Templates are represented with Node Templates that specify the occurrence of a specific service component within an application composition while relationships are represented with Relationship Templates that indicate the connecting elements and the direction of the relationship by defining one source and one target element.

Figure 2 Structural Elements of a Service Template and their relationships

A deployed service is an instance of a Service Template. A Service Template facilitates the composition of a service from various components and provides an abstraction that does not make assumptions about the hosting environments. Service Templates can be based on and built on-top of other Service Templates. The definition of non-functional behavior or quality-of-services are expressed in Tosca via policies. A Policy can express several diverse things like monitoring behavior, payment conditions, scalability, continuous availability, or security and privacy aspects. Sets of policies can be applied and associated with Node Templates and Relationship Templates each expressing the behavior that each instance of the Templates adheres to. In this sense, the OASIS Tosca specification is adequate enough to describe a Unicorn service graph that has been extensively described in previous Unicorn deliverables D1.1 and D1.2. A Unicorn service graph represents application compositions that follow the microservice architectural paradigm. Nodes of a service graph represent individual services while edges represent relationships between the microservices. It is also possible to apply design-time and run-time policies regarding the deployed applications that are represented on the service graph itself as attributes on either the nodes or edges of the graph. The concept of service graph


13

representation using the OASIS Tosca specification in Unicorn is given in more details and examples in Section 4.2 of this deliverable.

2.2 Docker Container Technology Stack Over the past few years, container technologies have been widely adopted by enterprises and SMEs, with significant benefits for DevOps teams. Such benefits include smaller memory footprints for user applications, faster deployment and bootstrapping times and improved isolation between containerized applications that share the same host resources. Among the multiple container technologies that have emerged, the one with the biggest market share and the widest adoption is Docker. In deliverable D1.2, Docker has been extensively described. In short, Docker is the industry standard container platform built on top of the linux kernel, namespaces, cgroups, chroot, and file system constructs. Docker provides additional tooling and a platform to manage the lifecycle of containers with the most notable ones including the Docker Engine, the Docker Swarm and the Docker Compose. The Docker Engine [10], whose diagram is depicted in Figure 3, is a client-service application and consists of three main components: i) the server which is long-running program called a daemon process (dockerd) ii) a REST API which specifies interfaces that programs can use to talk to the daemon and instruct it what to do and iii) a command line interface (CLI) client (docker command). The Docker daemon is responsible to listen for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. All requests performed by users to the Docker daemon are invoked on the Docker CLI layer using the docker command.

Figure 3 Abstract view of the Docker Engine

Furthermore, Docker Engine is responsible to run the docker containers which are constructed from docker images. An image is a read-only template with instructions for creating a Docker container namely the Dockerfile [11]. Often, an image is based on another image, with some additional customization. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image, which is then passed to docker daemon using he docker build command on the CLI for execution on the host. Apart from being able to simply run container images, a very useful feature for modern dockerized applications that follow the microservice architecture paradigm is the ability to orchestrate and fully manage the application lifecycle of containers. Although Docker does not support this feature out of the box, it can still offer such a functionality through Docker Swarm [12], which is a native tool for clustering. Swarm uses the standard Docker API to communicate with the underlying Docker daemon to manage containers on the host level. Contrary to single host container deployments, Swarm provides the ability to transparently scale to multiple hosts by turning a pool of Docker hosts into a single, virtual Docker host. Cluster management and orchestration features in Docker Engine 1.12 or later are available through the Swarmkit. Enabling Swarmkit in Docker Engine, allows it to run in swarm mode which focuses on micro-service architecture and supports service reconciliation, load


14

balancing and service discovery. Like most tools of the Docker technology stack, Docker Swarm follows the “swap, plug, and play” principle which means that it can be swapped out with a different backend as shown in Figure 4. The most popular and widely used are considered to be Apache Mesos [13] or Kubernetes [14].

Figure 4 Docker Swarm can be swapped out with different container orchestration technologies such as Kubernetes or Mesos

In addition to the aforementioned Docker toolset, another important tool provided by the Docker technology stack is Docker Compose [5]. Compose is a tool for defining and running complex applications consisted by multiple Docker containers. Multi-container applications can be defined in a single YAML file which is used to configure every composing service of the complex application (Figure 5 shows a simple docker-compose yaml file). A Compose YAML file defines services, networks and volumes. The main function of Docker Compose is the creation of applications that follow the microservice architectural paradigm, meaning the containers and the links between them. One major drawback of Compose is that it is limited within the scope of a single-host. In that sense, its concept is very similar to single-hosted Kubernetes pods [15]. In order to achieve multi-host deployment, additional container orchestration tools should be used such as Docker Swarm, Apache Mesos or a complete Kubernetes architecture as described above.

Figure 5 Docker-Compose that consists of 3 services; i) redis service which is bound onto frontend network, ii) db service that

mounts a volume on the host machine called db-data and bound on the backend network and iii) a vote service bound on frontend network


15

For Unicorn the main challenge is to offer a complete framework for the description of microservices enhanced with elasticity, security, monitoring and privacy policies definition. For this, we will rely on the extension fields offered by Compose. Those special fields can be of any format as long as they are located at the root of the Compose file and their names start with the x- character sequence. The contents of those fields are ignored by Compose, but they can be inserted at resource definitions as depicted in Figure 6.

Figure 6 x-custom special field definition. This field is not in the semantics of docker-compose language specification, however with

its definition it is being ignored or handled in a different manner by docker-compose.

2.3 Service Mesh and Container Design Patterns As a result of the constantly increasing trend of microservices and container adoption, the way application development and deployment is perceived has fundamentally changed, allowing new architectural patterns and technologies to emerge. This section, provides a more in-depth look in the topics of Service Mesh and the various Container Design Patterns. With the rise of adoption of container technologies as a popular mean to package and deploy microservices, the way distributed applications are built has been the focus of fundamental rethinking. Using containers enables developers to build their services using modular, reusable components, and this in turn leads to services that are more reliable, more scalable and faster to build than applications built using the monolithic approach. Similarly, to modern programming languages that use existing libraries to write new pieces of software, developers can compose together modular containers that are written by others which leads to thinking about symbiotic groups of containers that combined offer the full functionality set of a simple application service. Depending on the level of cooperation and the way those containers interact, they can be categorized into the following three Container Design Patterns [16]:

• The Sidecar pattern is the most common pattern for multi-container deployments. Sidecars extend and enhance the functionality of the main container. While it is always possible to build the functionality of a sidecar container into the main container, there are several benefits to using separate containers. Figure 7 depicts a nodejs container with a different container that syncs the file system with a git repository. Because of this modularity, the git synchronizer can be written once and be reused across numerous apps.

Figure 7 Example of a sidecar container that syncs the file system with a git repository


16

• The Ambassador pattern, illustrated in Figure 8 in which a secondary container proxy, called Ambassador, communicates to and from a main container. This container pattern simplifies container development in three ways: i) makes possible the simplification that the application only connects to a single server on localhost, ii) developers can test their application standalone and iii) they can reuse the ambassador with other applications that might even be coded in different languages. As an example, shown in Figure 8, consider a Redis cluster with read-replicas and a single write master. A Pod can be created that groups the main application with a Redis ambassador container. The ambassador is a proxy and is responsible for splitting reads and writes and sending them on to the appropriate servers. Because these two containers share a network namespace, they share an IP address and the application can open a connection on “localhost” and find the proxy without any service discovery.

Figure 8 Example illustrating the Ambassador design pattern

• The Adapter pattern presents the outside world with a simplified, homogenized view of an application. They do this by standardizing output and interfaces across multiple containers. The adapter pattern enables the heterogeneous world of legacy and open-source applications to present a uniform interface without requiring modification of the original application. The main container can communicate with the adapter through localhost or a shared local volume as shown in Figure 9 that depicts the task of monitoring N different applications. Each application may be built with a different way of exporting monitoring data. (e.g. JMX, StatsD, application specific statistics) but every monitoring system expects a consistent and uniform data model for the monitoring data it collects. By using the adapter pattern of composite containers, the heterogeneous monitoring data from different systems can be transformed into a single unified representation by creating Pods that groups the application containers with adapters that know how to do the transformation. Again, because these Pods share namespaces and file systems, the coordination of these two containers is simple and straightforward.

Figure 9 Example illustrating the Adapter design pattern

As already discussed, there are many patterns that symbiotic containerized applications adhere to, however being able to properly connect various services in cloud native environments may need a specialized networking model. To this end the term Service Mesh comes into play. A service mesh is a dedicated infrastructure layer for


17

handling service-to-service communication. It’s responsible for the reliable delivery of requests through the complex topology of services. In practice, the service mesh is typically implemented as an array of lightweight network proxies that are deployed alongside application code, without the application needing to be aware. One of the most popular frameworks that implements the service mesh is Istio [17]. Istio is a sidecar container implementation by Google for creating and managing microservices. Istio provides a complete solution to satisfy the diverse requirements of microservice applications by providing behavioral insights and operational control over the service mesh as a whole. An Istio service mesh is logically split into a data plane and a control plane. The data plane is composed of a set of Envoy proxies [8] deployed as sidecars that mediate and control all network communication between microservices, while the control plane is responsible for managing and configuring proxies to route traffic, as well as enforcing policies at runtime. Figure 10 depicts the two different panes that the Istio architecture consists of.

Figure 10 Istio Architecture

At Unicorn, we capitalize on the concepts of Sidecar Architecture and Service Mesh, by following good practices for inter-container communication across multiple clouds by using properly configured Envoy proxies within the Unicorn containers. Envoy is an L7 proxy and communication bus designed for large modern service-oriented architectures and is used to provide a consistent network topology for the Unicorn enabled microservices.


18

3 Unicorn Design Libraries

3.1 Overview The Unicorn Design Libraries are an essential part of the overall Unicorn framework. With the Design Libraries, developers are able to enhance the functionality of their microservices with concepts such as elasticity, monitoring, security, and privacy, concepts that as our survey in D1.1 [2] showed that developers do not have deep understanding and knowledge of although they are essential for their applications. For the Design Libraries to be less intrusive, their features and policies are offered as Java annotations. This way developers, at design-time, can annotate their code with a necessary concept while at run-time, the process to realize this concept is completely transparent to him/her. The aim of this section is to present analytically a formal meta-model described in EBNF format of the four Design Libraries that are offered as annotations at Unicorn design-time. The offered Design Libraries include: i) The Monitoring and Elasticity design library, ii) The Perimeter Security library, iii) The Risk and Vulnerability Assessment Library and iv) The Privacy and Authorization Library.

3.2 Monitoring and Elasticity

3.2.1 Design Library Overview The Monitoring and Elasticity Design Library aims at providing to the interested entities a toolkit for automating and managing the monitoring and scaling aspects of cloud applications deployed through multi-cloud containerized execution environments in an autonomic way. Through this library the user can define custom metrics and useful analytic insights concerning the runtime execution of an application. Additionally, the user can define elasticity policies to express the scaling behavior of the application that can ultimately improve the quality, performance and runtime cost. In Table 1, the definition of a Metric is described. A metric is an essential part of an autonomous system that aims to capture and quantify the behavior of a system’s element (e.g., how many requests are handled by a service). The unique identifier of a metric is composed by the name of the metric and the identifier of the agent that belongs to; each agent is responsible for collecting one or more metrics through its probes. The MetricDefinition production rule consists of useful information about the metric, such as, description, type, unit, group and an initial value. The attribute HigherIsBetter is useful for visualization and analysis purposes, as it specifies whether a higher value is better or not in terms of quality. The MetricConfiguration production rule, allows the use of configurations, such as controlling the intrusiveness and accuracy of various adaptive monitoring techniques. Table 2, describes the definition of the Monitoring Agent. An agent is composed from a set of probes and tags. The tags are useful for grouping agents that belong to the same tier (e.g., business-tier or presentation-tier). Each probe contains a set of metrics that are collected based on a periodical cycle (e.g., every 5 seconds) or when certain conditions are satisfied (event based). With the use of conditions, for example, an agent collects measurements only when the value is above or below a certain threshold (e.g., memory_usage < 5), or there is a change on its statistical properties (e.g., DIFF (cpu_usage) >= 2 ). The DIFF function denotes the difference between the value and the arithmetic mean of the timeseries stream in standard deviation units. The APTITUDE denotes the magnitude of change of the value since the last measurement.

Table 3, describes the definition of an Insight. An Insight is basically a composite metric, namely one that is composed from multiple metrics, via composition and/or aggregation. A user can define an insight by firstly specifying the observation period of her interest and then compose with basic arithmetic operations the


19

aggregated metric values exposed by a set of members. The Members production rule allows the user to specify from which agents the metrics are collected. The remaining tables provide a description of the Elasticity language, which allows the specification of high-level policies, namely Elasticity Policies, for the adaptation of a cloud application. An elasticity policy, described in Table 4, is composed by an elasticity trigger and an elasticity action. The policy specifies the action to be enforced when the policy is triggered. Note that, in case of conflicting policies, the priority value is used to resolve the conflict. An example of such policy is shown in Figure 11. In this example, the policy specifies that one more service of type svc-s (streaming service) should be added when there is i) a sufficient increase (10 requests/sec in that last 5 minutes) on the number of requests in the European region, ii) the aggregated cost of these services do not exceed the budget constraint (1.5 credits/hour), iii) and also there are less than 10 services running.

Figure 11: Elasticity Policy Example

A trigger, as described in Table 5, may contain multiple elasticity conditions, that should all be satisfied so that the policy is triggered. Each condition can be either a runtime information expression or a scheduled event. The Runtime Information expression can be composed from an existing insight metric or from an aggregated runtime information from resources, such as running instances, or hourly cost. The value of the runtime information is then compared with a user-defined value, in order to specify when the condition should evaluate to true in order to trigger the elasticity action, specified by the policy. In a similar fashion, a condition can be expressed using Scheduled Expression, specifying how often the condition should evaluate to true. This is particularly useful for taking scaling decisions based on traffic patterns. In Table 6, an Elasticity Action is described. Currently, the available actions refer to horizontal scaling, that is, replicating or removing resources. We should point out that a Resource is an abstract term and may refer either to a service or virtual machine flavor. The underline infrastructural resources are transparent to the Cloud Application Developer, as she only knows about the services of her application. The Unicorn platform is responsible about the infrastructural resources (e.g., virtual machines).


20

An Elasticity Action offers the ability to users and interested entities to specify a placement configuration for a service or an infrastructural resource in multiple clusters. A cluster can be located in the same zone or in another cloud provider, thus it supports multi-cloud scaling. Note that a placement configuration contains WarmupPeriod and CooldownPeriod production rules, which are important parameters for the efficiency of a scaling action and the overall performance of an application. The WarmupPeriod, is used to specify the time period that the metrics from newly instantiated resources are not considered by the Decision-Making & Auto-scaling service. Similarly, the CooldownPeriod is used to give time to the system to provision/de-provision new resources and absorb any changes, so as to avoid false scaling alerts.

3.2.2 Design Library Model Description

Table 1 Metric EBNF Definition

Metric ::= MetricInstanceID “:” MetricDefinition “,” MetricConfiguration

MetricInstanceID ::= “(” MetricName “,” AgentID “)”

MetricName ::= String

AgentID ::= String

MetricDefinition ::= Description Type Unit InitValue Min Max Group HigherIsBetter

Description ::= String

Type ::= “Float” | “Double” | “Integer” | “Boolean” | “String”

Unit ::= String

InitValue ::= Number | String

Number ::= [+-]? ([0-9]*[.])? [0-9]+

Min ::= Number | String

Max ::= Number | String

Group ::= String

HigherIsBetter ::= “TRUE” | “FALSE”

MetricConfiguration ::= Param (“,” Param)*

Param ::= Key Value

Key ::= String

Value ::= “Object”

Table 2 Monitoring Agent EBNF Definition


21

Agent ::= AgentID “:” Probes “,” Tags

AgentID ::= UUID

UUID ::= String

Probes ::= Probe (“,” Probe)*

Probe ::= Name “,” Metrics “,” CollectionEvent

Metrics ::= Metric (“,” Metric)*

CollectionEvent ::= Periodical | EventBased

Periodical ::= “COLLECT EVERY” PositiveInteger TimeUnit

PositiveInteger ::= [1-9]([0-9])*

TimeUnit ::= “Milliseconds” | “Seconds”| “Minutes”| “Hours”

EventBased ::= “WHEN” MetricConditions

MetricConditions ::= MetricCondition (“AND” MetricCondition)*

MetricCondition ::= MetricFunction “(” MetricName “)” RelOp Number

MetricFunction ::= “LAST_VALUE” | “DIFF” | “APTITUDE”

RelOp ::= “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE”

Number ::= [+-]? ([0-9]*[.])? [0-9]+

Table 3 Insight EBNF Definition

Insight ::= InsightID “:” CompositeMetricExpression “WITH” TimeWindow

InsightID ::= String

CompositeMetricExpression ::= MetricExpression | CompositeMetricExpression Operation MetricExpression

Operation ::= “+” | “-” | “*” | “/” | “DIV” | “MOD”

MetricExpression ::= GroupFunction “(” Members “)”

GroupFunction ::= “ARITHMETIC_MEAN” | “GEOMETRIC_MEAN” | “HARMONIC_MEAN” | “MEDIAN” | “P95” | “p99” | “COUNT” | “SUM” | “MIN” | “MAX” | “VARIANCE”

Members ::= MetricName “(” Agents “)” | MetricName “(” Agents “)” “WITH” Tags | MetricName “WITH” Tags

Agents ::= AgentID (“,”AgentID)*


22

Tags ::= Tag (“,” Tag )*

Tag ::= String

Table 4 Elasticity Policy EBNF Definition

ElasticityPolicy ::= ElasticityPolicyID “:” “When” ElasticityTriggerID “Perform” ElasticityActionID “With Priority” Priority

ElasticityPolicyID ::= String

ElasticityTriggerID ::= String

ElasticityActionID ::= String

Priority ::= PositiveInteger


Table 5 Elasticity Trigger EBNF Definition

ElasticityTrigger ::= ElasticityTriggerID “:” ElasticityConditions

ElasticityTriggerID ::= String

ElasticityConditions ::= ElasticityCondition (“,” ElasticityCondition )*

ElasticityCondition ::= RuntimeInfoExpr | ScheduledExpr

RuntimeInfoExpr ::= RuntimeInfo RelOp Number

RuntimeInfo ::= InsightID | ResourceInfo “(” Members “)”

ResourceInfo ::= “COUNT_RUNNING” | “COUNT_UNHEALTHY” | “HOURLY_COST”

Members ::= Member (“,” Member )*

Member ::= Resource | Resource “IN” Clusters

Clusters ::= ClusterID (“,” ClusterID )*

ClusterID ::= String

RelOp ::= “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE”

Number ::= [+-]? ([0-9]*[.])? [0-9]+

ScheduledExpr ::= “REPEAT_EVERY” TimeExpression

TimeExpression ::= Minute Hour DayOfMonth Month Weekday


23

Minute ::= [0-5][0-9] ( “,” [0-5][0-9] )* | “*”

Hour ::= ([0-1][0-9] | [2][0-3]) ( “,” ([0-1][0-9] | [2][0-3]) )* | “*”

DayOfMonth ::= ([0-2][0-9] | [3][0-1]) ( “,” ([0-2][0-9] | [3][0-1]) )* | “*”

Month ::= ([1-9] | [1][0-2]) (“,” ([1-9] | [1][0-2]) )* | “*”

Weekday ::= [0-6] (“,” [0-6] )* | “*”

Table 6 Elasticity Action EBNF Definition

ElasticityAction ::= ElasticityActionID “:” ResourceAction

ElasticityActionID ::= String

ResourceAction ::= ReplicationAction

ReplicationAction ::= “ADD” PlacementAddConfigs | “REMOVE” PlacementConfigs

PlacementAddConfigs ::= PlacementConfig “WITH” WarmupPeriod “,” CooldownPeriod (“,” PlacementConfig “WITH” WarmupPeriod “,” CooldownPeriod)*

PlacementConfig ::= PositiveInteger Resource “IN” Cluster


Resource ::= String

Cluster ::= String

WarmupPeriod ::= PositiveInteger TimeUnit

TimeUnit ::= “Milliseconds” | “Seconds” | “Minutes”| “Hours”

CooldownPeriod ::= PositiveInteger TimeUnit

PlacementConfigs ::= PlacementConfig (“,” PlacementConfig )*

3.3 Perimeter Security and Risk & Vulnerability Assessment

3.3.1 Design Library Overview The Perimeter Security and Risk & Vulnerability Assessment Design Libraries aim at providing an extra layer of security at the container level between intercommunicating Unicorn microservices (Perimeter Security Library) and the means to detect potential code vulnerabilities within the user developed microservices(Risk & Vulnerability Assessment Library). While there is a clear separation of Perimeter Security and Risk/Vulnerability assessment functionalities, we produced a uniform meta-model for covering their configuration and structuring of their reporting results. This enables the user to supply a single security model which covers both


24

functionalities. Furthermore, this solution still enables the user to provide separate security models, one per each functionality, based on his/her needs (or just one of them if the focus is just on a single functionality). As it has been already indicated, the focus is more on the configuration of the aforementioned security functionality. Such a configuration is given in a high-level of abstraction which is at the same level with respect to the way the user needs would be specified. In particular, the configuration enables the user to select different trade-offs between different parameters, like processing speed, accuracy and cost, which can then be mapped to more concrete terms into a respective optimal configuration of the security infrastructure. In addition, the configuration allows the user to choose from different rule sets in order to have a more focused analysis or protection. Such a selection can be in line with the application nature (and its components) and structure to also enable the saving of respective resources and costs. We should also note that the security model enables the reconfiguration of the application. This reconfiguration can take place in the form of: (a) certain security rules which when triggered lead to a reconfiguration of the security functionality (e.g., incorporation of additional rules to cover the protection though the IDS/perimeter security of a certain vulnerability); (b) direct request from the user which could be performed through the modification of the security model (e.g., to either modify his/her requirements or to point a new place were these requirements are specified and need to be enforced). The security model also covers the reporting of the results from the execution of the security functionality. Such a reporting is split into two different forms mapping to the two forms of functionality that are offered. Perimeter security reports focus more on describing a certain critical event taking place as well as on identifying the respective rule that has led to the generation of that event. On the other hand, vulnerability assessment reports focus on determining the overall application risk, the risk associated to the application components as well as the vulnerabilities that have been found across the whole application hierarchy. More details about the developed meta-model can be found in D4.1 [20]. In that deliverable, there is an extensive description of the meta-model coupled with a presentation of its conceptual model. In the following, we focus on supplying an exact form of the meta-model’s textual syntax while we also supply an example of a security model that can be specified in the context of a web-based application. Based on the textual syntax presented in Table 7, we need to supply the security model of a certain web-based PHP application. Further, suppose that the user has the following requirements for the two types of security functionality covered by this model:

• for the perimeter security, the user needs to have an optimal configuration according to the processing

speed and accuracy parameters - this means that the respective configuration should enable an as fast

as possible processing speed as well as an as accurate as possible analysis accuracy. He/she does not

care so much about cost, especially as he/she intends to place the perimeter security functionality at

the load balancer level. In addition, the user does not desire just to have identification of respective

issues but protection over them. Finally, logically speaking, the user focuses only on certain sets of rules

which provide detection & prevention over web applications that employ the apache web server.

• for the vulnerability assessment, the user requires to have a high accuracy as he/she desires to know

exactly what the true vulnerabilities are related to his/her application. He/she also requires employing

an internal scanning mode as such a mode can also supply the most extensive and accurate results

possible. The performance level of the scanning should be also high as the user does not desire that

his/her application is greatly influenced in terms of its performance for a huge amount of time. Thus,

the user prefers that scanning is performed quite rapidly without sacrificing accuracy of course. The

scanning access mode should be normal as the user does not desire to risk the accessing to critical


25

system files which could potentially enable a malicious intruder to have a control over the respective

resource hosting the user application. Finally, the scanning frequency should be once per week as the

application modification is actually performed on a weekly basis such that potential new vulnerabilities

could be introduced only after a one-week period each time.

Based on the above requirements, the actual security model that would have been provided by the user would

take the following form that conforms to the textual syntax that was presented above. The same information is

also depicted graphically in Figure 12 according to the abstract syntax of the security meta-model/language.

Figure 12 The security model conforming to the abstract syntax of the security meta-model

security model {

id 'WebAppSecModel'

perimeter security configurations{

perimeter security configuration{

id 'WebAppPerimeterSecurity'

response 'report plus fire rule'

vulnerability handling type 'IPS'

flavour 'speed+accuracy'

categories {'server-webapp', 'server-apache'}

location 'IDS at balancer'

}

}

risk assessment configurations{

risk assessment configuration{

id 'WebAppRiskAssessment'

response 'report to administrator'

frequency unit 'WEEKS'

frequency value 1

mode 'INTERNAL'

access mode 'NORMAL'

accuracy level 'HIGH'

performance level 'HIGH'

}

}

}

Code Snippet 1 Example for security and risk assessment configuration


26

3.3.2 Design Library Model Description The concrete syntax of the security meta-model (in EBNF form), overviewed in the previous sub-section, is supplied in the following table.

Table 7 Perimeter Security and Risk & Vulnerability Assessment EBNF

SecurityModel ::= “security” 'model' '{'

'id' String, ('security' 'configurations' '{' SecurityConfiguration ( "," SecurityConfiguration)* '}' )? ('perimeter' 'security' 'configurations' '{' PerimeterSecurityConfiguration ( "," PerimeterSecurityConfiguration)* '}' )? ('risk' 'assessment' 'configurations' '{' RiskAssessmentConfiguration ( "," RiskAssessmentConfiguration)* '}' )? ('security' 'reports' '{' SecurityReport ( "," SecurityReport)* '}' )? ('security' 'rules' '{' SecurityRule ( "," SecurityRule)* '}' )? ('vulnerabilities' '{' Vulnerability ( "," Vulnerability)* '}' )? '}';

SecurityReport ::= SecurityReport_Impl | RiskAssessmentReport | PerimeterSecurityReport;

SecurityConfiguration ::= 'security' 'configuration' '{' 'id' String 'risk' 'assessment' 'configuration' [RiskAssessmentConfiguration|String] ('security' 'rules' '(' SecurityRule|String ( "," SecurityRule|String)* ')' )? 'perimeter' 'security' 'configuration' PerimeterSecurityConfiguration|String '}';

PerimeterSecurityConfiguration ::= 'perimeter' 'security' 'configuration' '{' 'id' String 'response' Response 'endpoint' EString 'ids' 'type' IntrusionHandlingType 'flavour' Flavours 'categories' '{' categories IDSCategories ( "," IDSCategories)* '}' 'location' IDSLocations '}';


27

RiskAssessmentConfiguration ::= 'risk' 'assessment' 'configuration' '{' 'id' String 'response' Response 'frequency' 'unit' FrequencyUnit 'frequency' 'value' Int 'mode' ScanningMode 'access' 'mode' AccessMode ('component' 'names' '{' String ( "," String)* '}' )? 'accuracy' 'level' AccuracyLevel 'performance' 'level' PerformanceLevel ('schedule' Schedule|EString)? '}';

SecurityReportImpl ::= 'security' 'report' '{' 'id' String ('recipients' '{' String ( "," String)* '}' )? ('delivery' 'medium' DeliveryMediumType)? ('delivery' 'addresses' '{' String ( "," String)* '}' )? 'timestamp' Date '}';

SecurityRule ::= 'security' 'rule' '{' 'id' String 'constraint' String 'action' SecurityAction 'priority' Double '}';

Schedule ::= 'schedule' '{' 'id' String 'repetitions' Int 'start' 'time' Date 'end' 'time' Date '}';

Vulnerability ::= 'vulnerability' '{' 'id' String 'cve' String 'description' String ('counter' 'measures' String)? 'cvss' 'score' Double '}';

Response ::= 'report to administrator' | 'fire automated rule' | 'report plus fire rule';

IntrusionHandlingType ::= 'IDS' | 'IPS'; Flavours ::= 'speed+accuracy' | 'speed+cost' | 'accuracy+cost';

IDSCategories ::= 'server-webapp' | 'server-mysql' | 'server-apache' | 'protocol-ftp' | 'protocol-icmp' | 'policy-social' | 'os-windows';

IDSLocations ::= 'IDS per service instance' | 'IDS at balancer';

FrequencyUnit ::= 'HOURS' | 'DAYS' | 'WEEKS' | 'MONTHS' | 'YEARS';

ScanningMode ::= 'INTERNAL' | 'EXTERNAL' | 'BOTH';


28

AccessMode ::= 'NORMAL' | 'ADMIN' | 'BOTH'; AccuracyLevel ::= 'LOW' | 'MEDIUM' | 'HIGH';

PerformanceLevel ::= 'LOW' | 'MEDIUM' | 'HIGH';

DeliveryMediumType ::= 'EMAIL' | 'SMS';

RiskAssessmentReport ::= 'risk' 'assessment' 'report' '{' 'id' String ('recipients' '{' String ( "," String)* '}' )? ('delivery' 'medium' DeliveryMediumType)? ('delivery' 'addresses' '{' String ( "," String)* '}' )? 'timestamp' Date 'overall' 'score' Double 'app' 'name' String ('records' '{' VulnerabilityRecord ( "," VulnerabilityRecord)* '}' )? '}';

PerimeterSecurityReport ::= 'perimeter' 'security' 'report' '{' 'id' String ('recipients' '{' String ( "," String)* '}' )? ('delivery' 'medium' DeliveryMediumType)? ('delivery' 'addresses' '{' String ( "," String)* '}' )? 'timestamp' Date 'rule' 'category' IDSCategories 'violated' 'rule' SecurityRule|String '}';

VulnerabilityRecord ::= 'vulnerability' 'record' '{' 'id' String 'component' 'name' String ('vulnerabilities' '(' Vulnerability|String ( "," Vulnerability|String)* ')' )? '}';

SecurityAction ::= 'IDS_ALERT' | 'IDS_LOG' | 'IDS_PASS' | 'IDS_ACTIVATE' | 'IDS_DYNAMIC' | 'IDS_DROP' | 'IDS_REJECT' | 'IDS_SDROP' | 'IDS_RECONFIGURE' | 'RISK_ASSESSMENT_RECONFIGURE' | 'RISK_ASSESSMENT_START';

3.4 Privacy and Authorization

3.4.1 Design Library Overview Unicorn envisions to enhance the security aspects of the deployed applications by providing authorization mechanism that enables context-aware access control on the applications’ context. Through Unicorn we will


29

offer the ability to the developer to add code annotations and or configurations through deployment descriptor files in order to define the parts of the application that will be controlled by the context-aware access control. The whole annotation procedure and the annotation interpretation process should be a straightforward task, and the configuration through the deployment descriptor should be well documented. The purpose of logical access control is to protect from unauthorized access any type of resource such as data, services, executable applications, network devices or any other relevant object. Such access may include discovering, reading, creating, editing, deleting, and executing resources [21]. The access control protection is typically realized through a policy that describes which operations may be performed upon these resources, by whom, and in which context. If the subject that requests access to a resource satisfies the policy, then access is granted. If, on the other hand, the subject does not satisfy the policy, then access is denied. There have been several Access Control Mechanisms (a.k.a. ACM) that have been proposed in order to realize the various logical access control models. These models provide a framework and set of boundary conditions upon which the objects, subjects, operations, and rules may be combined to generate and enforce an access control decision. Each model has its own advantages and limitations. In Unicorn, the Attribute-based access control (ABAC) model has been selected mainly because of its generality and flexibility. For the sake of clarity, we will differentiate hereinafter in the deliverable the concepts of ABAC and ABAC Reference Implementations. Following the strict definition of NIST [22], we will refer to ABAC as the “access control method where subject requests to perform operations on objects are granted or denied based on assigned attributes of the subject, assigned attributes of the object, environment conditions, and a set of policies that are specified in terms of those attributes and conditions”. Following this definition, the terms “attribute”, “subject”, “object”, “operation”, “policy” and “environmental condition” will be unambiguous used based on the following definitions:

• Attributes are characteristics of the subject, object, or environment conditions. Attributes contain

information given by a name-value pair.

• A Subject is a human user or a systemic entity, such as a device, that issues access requests to perform

operations on objects. Subjects are assigned one or more attributes.

• A Resource (Object) is a system resource for which access is managed by the ABAC system, such as

devices, files, records, tables, processes, programs, networks, or domains containing or receiving

information. In fact, an object can be anything upon which an operation may be performed.

• An Action is the execution of a function at the request of a subject upon an object. Operations include

read, write, edit, delete, copy, execute.

• Policy is the representation of rules that makes it possible to determine if a requested access should be

allowed, given the values of the attributes of the subject, object, and possibly environment conditions.

• Environment conditions represent operational or situational context in which access requests occur.

Environment conditions are detectable environment characteristics that are modelled as attributes.

Environment characteristics are independent of the subject or object, and may include the current time,

day of the week, location of a user, or the current threat level.

For Unicorn, the XCAML (eXtensible Access Control Markup Language) standard will be used as the core of the context-aware authorization mechanism that Unicorn envisions. The Privacy and Authorization mechanisms that will provide the support to define Policy Enforcement Points and configure the authorization using ABAC, through XACML. More details about the usage of the developed model and also the development of the needed mechanisms can be found in D4.1 [20].


30

In the Figure 13 the ABAC model of Unicorn is presented; In Table 8, the details of the model are provided using EBNF format.

Figure 13 ABAC Model for Unicorn

3.4.2 Design Library Model Description In the following table, the model of the ABAC mechanism is descripted using the EBNF format.

Table 8 EBNF Definition of the ABAC model used in Unicorn

Request ::= RequestID, Action (“,”Attribute)*

Action ::= Resource , ActionType

ActionType ::= HttpVerb | GRPC | String

HttpVerb ::= “PUT” | “POST” | “GET” | “DELETE”

GRPC ::= String

Action ::= ResourceID (“,”Attribute)*

Environment ::= EnvironmentID (“,”Attribute)*

Resource ::= ResourceID (“,”Attribute)* (“,”Policy)

Policy ::= PolicyID “:Rule (“AND” Rule |“OR” Rule)*

PolicyID ::= UUID

Rule ::= RuleID “:” Expression (“AND” Expression |“OR” Expression)*

Expression ::= Attribute Operator Attribute

RuleID ::= UUID

Operator ::= “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE”

Attribute ::= AttributeID “:”Parameter


31

AttributeID ::= UUID

AttributeType ::= “Environmental” | “Resource” | “Request” | “

Parameter ::= Param (“,” Param)*

Param ::= Key Value

Key ::= String

Value ::= String | Number

Number ::= [+-]? ([0-9]*[.])? [0-9]+


32

4 Design Libraries Implementation

4.1 Design Libraries Implementation Aspects In Unicorn, the Design Libraries, assist developers to design cloud applications with minimum code intrusion required to enable portable security enforcement mechanisms, assure data privacy constraints, monitoring and elastic scaling across multiple cloud sites. At this early release phase, all the design libraries models are pretty generic and are implemented only for the Java programming language. Minimum code intrusion is achieved by annotating specific bits of code, methods or variables with decorators that provide additional functionality, without being a necessity for developer to have deep knowledge of the annotating concepts. The whole annotation and annotation interpretation process is transparent to the developer and does not require additional effort to be used. In the early release of the Unicorn Design Libraries, the functionality and features are offered through Java libraries that can be imported and called by developers at design-time with a mechanism pretty much identical to the dependencies and import mechanism offered by Maven [24], as depicted in Figure 14. It is worth mentioning that the design libraries, as it is discussed in Section 5.2, are pre-bundled within the development environment of the Eclipse Che IDE plugin, and Cloud Application Developers always have access to the latest version of the libraries.

<dependency>

<groupid>eu.unicornH2020 </groupid>

<artifactid>UnicornMonitoring</artifactid>

<version>LATEST</version>

</dependency> Figure 14 Maven Import of Unicorn Monitoring Library

For the implementation of the Unicorn Design Libraries annotations, Unicorn Developers implement JSR 250 – Common Annotations for the Java Platform [25], JSR 269 - Pluggable Annotation Processing API [26] and JSR 308 - Annotations on Java Types [27] . There are three methods that annotations are processed and interpreted that will be utilized in Unicorn. The first one is Source Code Generation. With this method annotation processing option works by reading the source code and generating new source code or modifying existing source code and for this reason Unicorn Developers use the Aspect Oriented Programming (AOP) [28] paradigm. The basic goal of AOP is to increase modularity by allowing the separation of cross-cutting concerns. It does so by adding additional behavior to existing code (an advice) without modifying the code itself, instead separately specifying which code is modified via a "pointcut" specification, such as "time the execution of this method" shown in Figure 15. This allows behaviors that are not central to the business logic (such as timeit) to be added to a program without cluttering the code, core to the functionality.

Figure 15 Timeit annotation that calculates the time needed for a method to be executed

The second method is Bytecode Transformation, which parses the class files with annotations and emits modified classes and newly generated classes. This method is used in order to automate the procedure of normative schema generation regarding the facets of metadata, requirements and configuration. To this end, specific types of annotations that will be processed during compilation will generate representative schema instances, which are in-line with the Unicorn Context Model.


33

import eu.unicornH2020.annotations.monitoring;

...

@UnicornMetric(name=”views”, handler=MetricHandlerType.CounterMetric,

units=””, valType=MetricValueType.Integer, initVal=0,

minValue=0, maxVal=Integer.MaxInt, higherIsBetter=true,

desc=”number of views for specific page”

)

public ItemController {

...

private int views;

...

//metric handler value extraction method

public int getViews() {

return views;

}

}

Figure 16 Example of Unicorn Monitoring Annotation

The final method is Runtime Reflections, which uses the Reflection API to programmatically inspect the objects at runtime and is used to dynamically implement specific behaviors that relate to programming interface binding, measurement of performance metrics and runtime configuration such as the example shown in Figure 16. In this example the developer creates a metric called views, that on runtime execution it will dynamically search and be bound to the getViews() method that acts as the metric handler.

4.2 Unicorn Service Graph Representation Unicorn from its early stages of conceptualization till its current state in this specific time has adopted the notion of the Service Graph. As all known graphs, a Service Graph at its core is defined as a set of nodes and edges connecting them. With the Service Graph, developers are able to describe the topology of applications since the nodes of the graph represent the various services while edges represent the relationships between them. What is more to it is the fact that edges in a Service Graph are directed denoting the direction of flow between data of the participating services. Additionally, nodes are described by a number of attributes denoting resource management parameters (e.g., VCPUs, RAM, disk type, disk size, Network interfaces, etc.), monitoring metrics, cost constraints and elasticity policies. Figure 17, showcases a sample streaming application consisting of 3 services depicted as nodes; a front-end, a back-end and a user service. As previously mentioned, each node is additionally described by a set of attributes that enhance the microservices with features offered by Unicorn and dictate deployment and initial placement and configuration preferences. As an example, for the above, the front-end service is deployed using the image front_end and it is deployed in two different zones. At the eu-1 zone it instantiates three replicas and at the us-1 it instantiates 1 replica among other configurations. In a similar manner the configuration of properties of the rest of the services can be interpreted. Furthermore, this Service Graph indicates that the front-end service exchanges data with the users service and the backend, while the backend communicates with the users service.


34

Figure 17 Simple Service Graph with properties

At Unicorn the Service Graph is being created and edited at the Management Perspective of the Unicorn Dashboard which is more thoroughly explained in Section 5. There, Unicorn offers an intuitive drag and drop modelling tool, build using Javascript web technologies, to ease topology description. Simultaneously, this intuitive visual representation is translated into an intermediate formal representation of the Service Graph using the YAML serialization of the OASIS Tosca specification. Using this convention, each node of the Service Graph is represented as a Node Template while the relationship is mapped to Tosca’s Relationship Templates. Properties, policies, configuration and features are defined using the properties tag of the specification syntax. Figure 18 illustrates the Tosca YAML topology description of the above Service Graph.


35

Figure 18 Representation of Service Graph as Tosca YAML

At the end phase of the deployment functionality, a Docker-Compose file is produced and passed to the next components of the Unicorn Platform to continue with the process of deployment. Compose is a tool for defining and running complex applications consisted by multiple Docker containers. In Unicorn, service composition consists of dockerized microservices that are enhanced with elasticity, security, monitoring and privacy functionality, by using pre-compiled Unicorn Docker Images. The main components that consist a Docker-Compose description are services, networks and volumes. It is obvious that Docker-compose file components are easily mapped to Tosca nodes and relationships, thus making the representation of the Tosca service graph and Docker-compose service description conceptually equal.

version: "3.3" services: front_end: image: front_end:latest ports: - "8000:80" back_end: image: back_end:latest users: image: users:latest x-resources: front_end: type: front_end properties: zones:


36

- eu-1: replicas: 3 resources: cpu: 2 ram: 2 - us-1: back_end: type: back_end properties: zones: - eu-1: replicas: 2 resources: cpu: 4 ram: 8 - jp-1: replicas: 1 resources: cpu: 2 ram: 4 users: type: users properties: zones: - eu-1: replicas: 1

Code Snippet 2 Docker-Compose description of the service graph presented above

4.3 Unicorn Compose File The Unicorn Docker Compose file is an extension of the Docker Compose file where we add extra rules to support the features that Unicorn framework will provide, such as security, monitoring, balancing etc. For the creation of the Unicorn Compose files, the Docker Compose Schema version 3.31 has been used as basis and extended. Docker Compose can use both YAML and JSON, but for the Unicorn Compose Parser we selected to support YAML description files, as it allows easier understanding of the hierarchy of the defined model and is a superset of JSON. Furthermore, we decided to add the Unicorn rules as extra YAML root rules and not to perplex the well-defined Docker Compose rules, as explained in section 2.2. In the following listing we represent an example the Unicorn Compose file model description.

version: “3.3”

services:

serviceName:

image: imageName:imageTag

ports:

- “hostPort1:containerPort1”

- “hostPort2:containerPort2”

volumes:

- hostVolumePath1:containerVolumePath1

1 https://github.com/docker/compose/blob/master/compose/config/config_schema_v3.3.json

https://github.com/docker/compose/blob/master/compose/config/config_schema_v3.3.json


37

- hostVolumePath2:containerVolumePath2

networks:

- networkName

... other docker compose service rules

networks:

networkName:

driver: driverName

... other docker compose root rules

x-balance:

... balance rules

x-security:

firewall:

... firewall rules

x-monitoring:

... monitoring rules

x-scalability:

Code Snippet 3 Example of Docker Compose file

The full version of the Unicorn Compose file, as well as an extended example are provided in the document’s Annex.

4.3.1 Unicorn Compose Descriptor Interpreter Unicorn Compose Filer Interpreter (Error! Reference source not found.) is responsible for the interpretation of the Unicorn Compose file to specific actions that the orchestrator will be able to execute. A dedicated component has been created for this reason and has been added to the Unicorn GitLab repository2. This component has been created using Java and Spring Boot and can be separated to three main sub packages, as depicted in Figure 19, that oblige different purposes of the interpretation process. The first package is the Entities package, which is used to model the Unicorn Compose rules into java objects. The other two packages are the Parsers package and the Writers package. Parsers includes Classes responsible for parsing the Unicorn Compose file and construct a Unicorn Compose java object, while Writers allow to write back the Unicorn Compose java object to a YAML file with whatever modifications we have done on it. For the development and the extending of the Unicorn Compose Interpreter we decided to follow the Unicorn Compose file hierarchy for each of the packages, thus putting the Docker Compose specific rules into one package and for each other root Unicorn specific rule we create a different root package. As depicted in the figure bellow, for each Docker Compose root rule a different java file is used. The only exception, due to its complexity, is the services root rule, where we have to create a ServiceEntities package, for the Entities package only, and not just one java file in order to construct more appropriately the subEntities corresponding to the service. For the Unicorn specific rules, same as before, we create a root package for each root rule where the respective java file is put

2 https://gitlab.com/unicorn-project/


38

Figure 19 The Unicorn Descriptor Interpreter

4.4 Unicorn Enabled Container Packaging All Unicorn enabled microservices before deployment to the available cloud offerings need to be packaged within Docker containers. For a Docker container to be Unicorn enabled and be able to offer the desired Unicorn functionality, it has to have bundled specific execution environment agents and processes that are enlisted in Table 9. The enablement of these agents and processes, is part of Unicorn Continuous Integration and Delivery (CI/CD) cycle where for each instantiated containerized environment that hosts a service of a cloud application, the Unicorn Smart Orchestrator configures, bundles and deploys alongside the application, a Monitoring Agent,


39

Monitoring Probes, the Envoy Proxy, the Envoy Configuration file and the Snort IDS with the accompanying scripts from a publically accessible Unicorn repository. In regard to configuration, users can provide their own configuration parameters for the bundling processes, that are passed to the immutable container execution environment as container environmental variables.

Table 9 Docker container requirements for Unicorn enabled microservices

Container Agents & Processes Description

Monitoring Agent This agent is responsible for the collection and dissemination of application and infrastructure metrics. The collection of metrics is a task assigned to dedicated lightweight background processes, the probes.

Monitoring Probes Monitoring Probes, whom model has been described in Table 2, are responsible for collecting specific metrics from inside the container runtime and application performance measurements. Such monitoring probes include probes for measuring CPU, Memory Utilization, Latency, Method execution time etc.

Snort IDS Snort [7] is a state-of-the art IDS whose main task is to allow or deny incoming traffic based on user defined policies. It is also responsible to raise alerts and notify users when abnormal network traffic and behavior is detected.

Snort Configuration Scripts Snort Configuration Scripts are configuration files that configure Snort to run in one of the following three modes: i) Sniffer Mode, ii) Packet Logger Mode, iii) Network Intrusion Detection System

Envoy proxy The Envoy proxy is responsible to handle all incoming and outgoing network traffic. User developed service uses localhost to communicate only with the Envoy proxy, thus implementing the sidecar architecture pattern.

Envoy Proxy Configuration File The Envoy Configuration files is a file in Json or Yaml format that provides the necessary configuration parameters to the Envoy Proxy to manage incoming and outbound traffic between microservices

For these requirements to be met and the agents and processes to be bundled within Docker containers, specific steps have been taken in order to create a set of Unicorn parent container images. A Unicorn parent image is the image that the Unicorn enabled image are based on. The first step is the compilation of the Unicorn Dockerfile. As already mentioned in Section 2.2, a Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. A Dockerfile consists of the following elements:

• The FROM instruction which sets the base image for the container to be built. Unicorn offers parent images that are based on i) an Ubuntu 16.04 Xenial [29] images which is a widely used docker base image, ii) Alpine image [30] which is a lightweight linux based base image with small memory footprint and iii) An RPM based, Centos:latest base image

• The ADD or COPY instructions that copy files from the host machine to the container’s filesystem. Using numerous ADD or COPY instructions in the Dockerfile, the agents, processes and probes presented in Table 9 are added within the Docker container.


40

• The RUN set of instructions will execute any commands in a new layer on top of the current image and commit the results. The resulting committed image will be used for the next step in the Dockerfile. The purpose of these instructions is mainly to install a Java 8 JDK in order to be able to execute the Monitoring Agent and the Monitoring Probes inside the container since they are built with Java.

Figure 20 illustrates an example of a Dockerfile which when compiled produces a Unicorn parent image based on Ubuntu Xenial. It should be noted that all file names in COPY instructions are indicative.

Figure 20 Example Dockerfile that builds a Unicorn parent image based on Ubuntu 16.04

The second step of the process is the actual building of the base image by invoking the docker build command of the Docker CLI. Finally, the last step of the process is to push the newly created Unicorn base image on a public Docker image repository such as Dockerhub [31] or a privately hosted one.

4.5 Unicorn Design Libraries Features

4.5.1 Monitoring and Elasticity Design Libraries Features

Table 10 Monitoring and Elasticity Table of Features

Feature/Policy Description Provided as Annotation

Define custom metrics

Define a custom metric for a service, specifying useful information such as, collection intensity, description, measurement units, etc.

Yes

Metric Configuration

Change the configuration of a metric (e.g., the intensity level)

Design Time: Yes, Runtime: No

Define Insight metrics

Define an insight metric which is composed from multiple aggregated metrics over the available resources.

No

Scale based on service behavior using thresholds

Through an Elasticity Policy a service can be scaled based on its behavior which is captured by monitored data (analytic insights) such as, resource utilization and performance metrics.

Yes

Acknowledge cost constraints

The elasticity policy can adhere to cost constraints regarding the execution of the cloud application

No

Scale on multiple cloud providers

Through the elasticity actions, services can be (de-) provisioned to subscribed cloud providers and regions.

Yes


41

Scale based on traffic patterns

Scale a service based on traffic patterns (e.g., daily or seasonal patterns) by scheduling scaling actions at specified times.

No

4.5.2 Perimeter Security Design Library Implementation

Table 11 Perimeter Security Table of Features


Location The location for placing the IDS. It can be either at each user VM or at the load balancer level

Yes

Categories The sets of (detection/prevention) rules mapping to certain categories which focus on different aspects or kinds of applications or technologies/components employed for these applications. Currently, 7 categories are supported, namely: ‘server-webapp’, ‘server-mysql’, ‘server-apache’, ‘protocol-ftp’, ‘protocol-icmp’, ‘policy-social’, ‘os-windows’

Yes

Flavour The actual optimisation flavour to be selected for configuring the perimeter security infrastructure. Each flavour attempts to optimise two out of three possible parameters. The flavours covered are thus the following: ‘speed+cost’, ‘speed+accuracy’, ’accuracy+cost’

Yes

Intrusion Handling Type

Here we actually explicate the mode of operation of the perimeter security solution. In particular, such a solution can be either in detection or also prevention mode. So, the corresponding options are the following: ‘IDS’ & ‘IPS’, respectively.

Yes

Response Three types of response can be produced by the perimeter security functionality in case the detection of an issue takes place: (a) report to administrator; (b) fire a certain security rule; (c) perform both actions. Obviously, when the Intrusion Detection Solution is IDS, only (a) makes sense as an option. When it is IPS, then (b) or (c) make sense.

Yes

4.5.3 Risk and Vulnerability Assessment Implementation

Table 12 Risk and Vulnerability Assessment Table of Features


Access Mode Scanning can be performed in two possible modes: (a) NORMAL where the scanning tool has access to non-private files or resources of the application; (b) ADMIN where the scanning tool has access to private resources, like OS-based ones

Yes

Scanning Mode Scanning can be performed either remotely, internally or in both ways. A remote scanning might not be able to unveil all possible vulnerabilities while the confidence for some of them might not be ideal. On the other hand, internal

Yes


42

scanning has the potential to discover more vulnerabilities with higher confidence but leads to the spending of precious application resources. A hybrid scanning mode might lead to delivering the greatest amount of vulnerabilities, but it is the more complex to support while still suffering from the resource consumption drawback of the internal scanning.

Accuracy Level The accuracy of the scanning can be LOW, MEDIUM or HIGH. Each accuracy level maps to a certain level of resource consumption and cost. For instance, a LOW accuracy might require employing a small set of vulnerability scanning rules such that the resource consumption is low and the scanning performance is optimal. On the other hand, a HIGH accuracy might require employing an extensive rule set which would then lead to high resource consumption and medium or bad scanning performance (e.g., taking a great amount of time to complete just one vulnerability scanning).

Yes

Performance Level

Similarly to accuracy, the same values can be alternatively supplied by the user. This means that LOW performance means the scanning will take longer to execute but it might be lighter in term of resources. On the other hand, a HIGH performance means that the scanning should be fast which requires employing a great amount of resources. Combined with the previous feature, the user could explore different possible trade-offs between them which could lead to different configuration forms of the respective vulnerability assessment infrastructure. For instance, putting accuracy and performance as HIGH means that a great amount of resources would need to be devoted. This would then require elaborating the actual interference to the user application and keep it to the minimum. It could, for instance, map to using a very efficient scanning tool in remote mode such that the actual user application infrastructure is not impacted by the respective scanning. Thus, as it can be seen, the setting of both features can have a great impact on both the way the scanning can be performed as well as the amount of resources that could be devoted to it.

Yes

Frequency Unit This feature explicates the unit of frequency of the scanning. As such, the following units can be alternatively supplied: HOURS, DAYS, WEEKS, MONTHS or YEARS. The frequency might be influenced by the actual risk in vulnerability exploitation as well as the frequency via which an application evolves. If the exploitation risk is then high or an application evolves rapidly (such that the application evolution leads to opening some vulnerabilities), this means that the scanning frequency should be more fine-grained. On the other hand, if the exploitation risk is small and the application evolution is

Yes


43

less frequent, the vulnerability scanning can also be less frequent.

Frequency Value This feature signifies the actual frequency value in association to the previous feature. For instance, if the value is 1 and unit is DAYS, this means that the scanning can be performed every day. On the other hand, if the value is 6 and unit is HOURS, then this means that the scanning frequency will be every 6 hours.

Yes

Response This is the same field as in the case of Perimeter Security configuration. In this respect, the outcome of the vulnerability scanning can be either reported to an administrator, lead to the execution of a security rule (e.g., could map to increasing the protection level of IPS due to a discovered vulnerability), or mapped to performing both previous actions.

Yes

4.5.4 Privacy and Authorization Implementation

Table 13 Privacy and Authorization Table of Features


Define Policy Enforcement Point (PEP) at the level of service

Enable context aware access control to a service of the service graph

No

Define Policy Enforcement Point (PEP) at the level of class or method

Enable context aware access control on an action of a service

Yes

Definition of the policies, rules and expressions

User defines the needed expressions, rules and policies that will be used for the decision making of the ABAC

No

5 Unicorn Dashboard The Unicorn Dashboard is the focal point of all user interactions, including development and administrative actions, with the underlying Unicorn Platform and the Cloud Execution Environment. Through the Dashboard’s unified development and management environment, application developers build their applications following the microservice architectural paradigm, package them as Docker containers and deploy them to the cloud, while application administrators manage their applications’ lifecycle, monitor deployed application and formulate privacy, elasticity and security policies that govern their deployments with minimum effort. Even though Unicorn Dashboard is a unified environment, it is obvious that there is a logical distinction between functionalities provided to different Unicorn user roles, mainly to application developers and application administrators. This functionality separation is reflected at the implementation of the Dashboard layer, which is divided into two perspectives, i) The Management Perspective and ii) The Development Perspective illustrated in Figure 21.


44

Figure 21 The Dashboard layer divided into two perspectives.

5.1 Unicorn Management Perspective The Unicorn Management Perspective is realized by the Unicorn Dashboard. The Unicorn Dashboard serves as web frontend of the Unicorn platform such that one aspect of the Dashboard is the on-boarding of new Unicorn users. Therefore, the access and registration of users is executed in the Dashboard. As the Dashboard realizes the Management Perspective of Unicorn, the Dashboard will be realized with focus on administrative and managing purposes, e.g. managing their applications’ lifecycle, monitoring of deployed applications and, moreover, managing Unicorn users according to their dedicated roles. For management tasks as the lifecycle management, the Dashboard allows users to interact directly with the Unicorn platform, and therefore with the developed microservices for Unicorn. Every micro-service exposes their own REST API such that the developed frontend, the Dashboard, will retrieve, update, create and delete data in the Unicorn platform by interacting with the microservices through a RESTful interface. Registration and administration of Unicorn users is the other aspect of the Unicorn Dashboard. Users, asking for registration, are cloud application administrators, developers or, also, cloud providers. The user management will take care of registration and administration of users in the Unicorn Dashboard. The development perspective will be built on top of Eclipse Che. In order to grant access to the Unicorn Che Stack directly the Unicorn Dashboard offers the possibility to launch directly from the Dashboard.

5.1.1 Entities and Entity Relationship in Unicorn Dashboard

As described above the Management Perspective realized within the Unicorn Dashboard tackle management and administrative purposes. In Figure 22 the main entities of the Dashboard are depicted and will be discussed in the following.


45

Usergroup

ugroupID

name

User

UUID

username

password

Application

appID

name

owner

Monitoring

monID

name

agents

Offering

offeringID

name

provider

usergroup

company

email

firstname

lastname

role

policies

monitoring data

monitoring metrices

price

metrics

Figure 22. Entities in Unicorn Dashboard.

Usergroups As an initial approach usergroups can be associated with companies working on cloud-based software solutions. To group users into a usergroup is necessary because there can be users with different roles but associated with the same application. As an example, there can be a cloud application developer who has developed the application and an assigned cloud application administrator who is responsible to manage the app lifecycle. But, the developer also needs to have access to the monitoring data of the deployed application in order to react as fast as possible to suspicious monitoring data. Therefore, the access rights of a single user may need to be copied to the usergroup the user belongs to. A usergroup contains the unique ID and the name of the group as attributes. At this initial state a user can only be assigned to one user group. In a further step we can envision that there are several software projects within one company and different members are working on several projects. This extension gives some complexity to the Unicorn Dashboard that will be considered in next steps. User The users are the actors in the Unicorn platform. They belong to the roles defined in D1.1, and especially, in the frame of the Unicorn Dashboard

• Cloud Application Owner

• Cloud Application Product Manager

• Cloud Application Developer

• Cloud Application Administrator

• Cloud Provider

• Unicorn Developer

• Unicorn Administrator

The User entity contains the attributes required to identify a user, e.g. a UUID, username, full name, company and usergroup, contact data and user role. Furthermore, access data for Unicorn is also contained within the entity User.


46

Application Applications are entities belonging to assigned users and, therefore, also to corresponding usergroups. The entity application is created through the developer perspective of Unicorn or directly by the user-owner. It contains the unique appID, the application name, defined application policies with respect to security and privacy as well as user defined monitoring metrics. Also, the produced monitoring data from the assigned monitoring agent belongs to the application. Monitoring Monitoring Data is created through monitoring agents assigned to Unicorn-compliant applications. Monitoring data can be retrieved directly from the Unicorn Dashboard by the users owning the monitored application. The monitoring entity in the Unicorn Dashboard contains an ID, the name, the corresponding metrics and monitoring agents. Offering The entity offering in the Unicorn Dashboard is needed to search for a fitting offering to deploy the application with respect to the defined application properties. Therefore, the entity offering belongs to a specific cloud provider and contains next to the offeringID the offering name, corresponding cloud provider and the price. The relation between the different entities is already described in the entity descriptions. A general overview of the relationship between the entities in the Unicorn Dashboard is depicted in Figure 23.

Usergroup User

UUID

Application

appID

Monitoring

monID

Offering

offeringID

ugroupID

is member

has app

is monitored

is deployed on

Figure 23. Entity Relationship in Unicorn Dashboard.

5.1.2 Technical Approach

Unicorn follows the micro-service approach. The Dashboard implementation also reflects and supports this approach. Furthermore, the Dashboard itself is also considered as a micro-service. All microservices included in the Unicorn platform expose their own REST API through which data can be retrieved, updated, created and deleted. There exist different approaches for developing UI; we distinguish between server-side UI frameworks, e.g. Thymeleaf [32], and client-side UI frameworks as Angular [33]. In a microservice architecture, a server-side UI framework implies that every micro-service needs to bring their own UI component while a client-side UI framework can use the exposed REST API of the microservices without any UI component on the server-side. Angular is an open-source Typescript-based UI framework supported by Google. As a client-side framework it profits from an increased velocity and high usability. Based on Typescript and, therefore, on the newest generation Javascript, Angular includes ECMAScript6. This leads to high scalability and high extensibility of the


47

developed UI such that the initial Unicorn Dashboard can be extended easily without loss of performance. Another aspect is the possibility to create an automated test system to test the UI code which leads to high code quality. Together with a modular and object-oriented approach the development of the Unicorn Dashboard can follow a fast and agile concept. Nevertheless, server-side UI approaches are also still relevant in developing UI. Thymeleaf is a template engine, lightweight and high performant without a complex compiling process. The advantage of Thymeleaf is furthermore the availability of diverse UI templates looking modern and fresh. Also, the integration with well-established frameworks as Spring MVC [34] and Spring Security is an advantage. Therefore, the Unicorn Dashboard development will start with an Angular based UI development. Due to the advantage of providing a common UI layout using Thymeleaf as a template engine in a next step the Unicorn Dashboard can be enhanced with Thymeleaf template engine.

5.1.3 Provided Functionalities

In the following section we discuss the functionalities of the Unicorn Dashboard in more detail. In this discussion we describe the components of the Dashboard itself but also the needed interaction with different microservices of the Unicorn platform and their APIs.

5.1.3.1 User Management

The User Management is built as a micro-service using Spring Boot [35] and Java. For software project management and build system we use Apache Maven. The database for user management is set up using MySQL. Furthermore, the database is containerized with Docker and we use Docker compose to deploy the database. The User Management in the Unicorn Dashboard encompasses all functions related to Unicorn users, starting from registration until editing existing users. Also, the access control is integrated in the User Management. This is realized by integrating the established Spring Security [36] framework. The following Tables describe the User Management functionalities offered by the Management Perspective of the Dashboard.

Table 14 User Registration Functionality

Functionality Name

User registration

Related Component

User Management

Description By adding the respective details, the user can be registered within Unicorn platform. A new entry is added into the User table in the database.

HTTP Method POST

Table 15 Login/Logout Functionality

Functionality Name User Login/logout

Related Component Authorization, User Management

Description By entering credentials the user is logged into Unicorn platform. Clicking on Logout button the user is logged out.

HTTP Method POST


48

Table 16 Delete User Functionality

Functionality Name

Delete User

Related Component

User Management

Description User can be deleted from Unicorn platform. The corresponding dataset is deleted from the User table in the database.

HTTP Method DELETE

Table 17 Update User Profile Functionality

Functionality Name

Update User

Related Component

User Management

Description User details which are saved in the User table can be updated with this function. The corresponding dataset is also updated and saved in the User table of the database.

HTTP Method PUT

Table 18 Change Authorization Details Functionality

Functionality Name

Change authorization details

Related Component

Authorization, User Management

Description The authorization data corresponding to the user is changed. The new credentials are saved in the database.

HTTP Method PUT

Table 19 Register a new Usergroup Functionality

Functionality Name

Register Usergroup

Related Component

User Management

Description Create and register a new usergroup within Unicorn. HTTP Method POST

Table 20 Update an existing usergroup Functionality

Functionality Name

Update Usergroup

Related Component

User Management


49

Description The attributes of an existing usergroup can be changed. The new attributes are saved within the corresponding dataset in the database.

HTTP Method PUT

5.1.3.2 Application Management

The application management is a UI component which allows the management of Unicorn compliant applications. The following Tables describe the Application Management functionalities offered by the Management Perspective of the Dashboard.

Table 21 Start a Unicorn-compliant app Functionality

Functionality Name

Start Application

Related Component

App lifecycle management

Description An application is started through the Unicorn Dashboard. The corresponding Docker run command is triggered to start the container.

HTTP Method POST

Table 22 Stop an application Functionality

Functionality Name Stop Application

Related Component


Description An application is stopped by triggering the Docker command to stop the running container.

HTTP Method POST

Table 23 Delete application Functionality

Functionality Name

Delete Application

Related Component


Description An application is deleted from the Unicorn Dashboard by deleting the corresponding Docker container.

HTTP Method DELETE

Table 24 Deploy Unicorn Compliant Application Functionality

Functionality Name

Deploy Application

Related Component

App Lifecycle Management, Deployment Enforcement


50

Description After setting up the deployment environment for an application the application can be deployed through Unicorn Dashboard. The corresponding Docker command is triggered.

HTTP Method POST

Table 25 Get Monitoring Data Functionality

Functionality Name

Get monitoring data in real-time

Related Component

Monitoring Service, Real-Time Notification, Vulnerability Assessment Service

Description Monitoring data retrieved from monitoring service is visualized in real-time. Retrieved data depends on defined monitoring metrics and activated monitoring agents.

HTTP Method GET

Table 26 Update Runtime Policies

Functionality Name Update runtime policies

Related Component

Runtime policies & SLA Enforcement

Description Runtime policies can be updated through Unicorn Dashboard. New policies are invoked.

HTTP Method PUT

Table 27 Update Service Graph Functionality

Functionality Name

Update service graph

Related Component

Resource Manager

Description An updated Docker Compose file is sent to the Unicorn platform. The new service graph will be generated by the Resource Manager.

HTTP Method POST

Table 28 Cloud Offering Marketplace Functionality

Functionality Name

Cloud Offerings Marketplace

Related Component

Cloud Offerings Manager

Description All available cloud offerings in Unicorn platform are listed. Details regarding offering name, prices, etc. can be retrieved in the detail view.

HTTP Method GET


51

5.1.3.3 Orchestration and Resource Management

Orchestration and resource management for Unicorn-compliant applications is also located within the Unicorn Dashboard. The following tables describe the Application Management functionalities offered by the Management Perspective of the Dashboard.

Table 29 Generate Service Graph Functionality

Functionality Name

Generate Service Graph

Related Component

Resource Manager

Description The service graph for an application is generated by sending the corresponding Unicorn-enhanced Docker Compose file to the Unicorn platform.

API Call POST

Table 30 Update Service Graph Functionality

Functionality Name

Update service graph

Related Component

Resource Manager

Description An updated Docker Compose file is sent to the Unicorn platform. The new service graph will be generated by the Resource Manager.

API Call POST

5.1.3.4 Launch Unicorn IDE Plugin

The Development Perspective of the Unicorn Dashboard is built on top of the open-source cloud IDE EclipseChe. The Unicorn Che Plugin allows the implementation of Unicorn compliant web applications. To grant easy access to the Unicorn Che Plugin the Unicorn user is able to launch Eclipse Che directly from the Unicorn Management Perspective of the Dashboard.

Table 31 Launch Unicorn IDE Plugin

Functionality Name Launch Unicorn Eclipse Che

Related Component Eclipse Che

Description The user is directed to a Unicorn enhanced Eclipse Che

5.1.4 Exposed APIs In this section the exposed APIs of the services described above are described. In general, the services of the Unicorn Dashboard are UI services consuming APIs from the micro-services developed to build the Unicorn Dashboard. However, the described User Management as well as the Authentication component are backend services with exposed APIs.


52

Table 32 Update Usergroup API

Description Update an Usergroup

Endpoint /Usergroup/:ugroupid/

HTTP Method PUT Note Password can be changed by using the authorization service

Query Parameters ugroupid

Request/Response Format json/json Sample Request /usergroups/29bc6362-22d5-43e7-906a-88df8c65e212

Sample Request Body {name: changedSMEorganizationname}

Sample Response Body {name: changedSMEorganizationname uuid: 29bc6362-22d5-43e7-906a-88df8c65e212}

Table 33 Delete Usergroup by ID API

Description DELETE usergroup by ugroupid

Endpoint /usergroup/:ugroupid/

HTTP Method DELETE

Note

Query Parameters ugroupid

Request/Response Format json/-

Sample Request /usergroups/29bc6362-22d5-43e7-906a-88df8c65e212

Sample Request Body {name: SMEorganizationname uuid /29bc6362-22d5-43e7-906a-88df8c65e212 }

Sample Response Body -

Table 34 Update User API

Description UPDATE an User

Endpoint /user/:uuid/

HTTP Method PUT

Note uuid is generated automatically, password is transferred and saved encrypted

Query Parameters uuid

Request/Response Format json/json

Sample Request /user/d851c459-9e97-4760-b929-153da7d49665

Sample Request Body {username: changeduser firstname: example lastname: user email: [email protected] company: SME}

Sample Response Body {uuid: d851c459-9e97-4760-b929-153da7d49665 username: changeduser firstname: example lastname: user email: [email protected] company: SME}

mailto:[email protected]



53

Table 35 Delete User API

Description DELETE an User

Endpoint /user/:uuid/

HTTP Method DELETE

Note


Request/Response Format uuid/- Sample Request /user/d851c459-9e97-4760-b929-153da7d49665

Sample Request Body {uuid: d851c459-9e97-4760-b929-153da7d49665}


Table 36 Get all Users API

Description GETALL user Endpoint /user/

HTTP Method GET

Note - Query Parameters Ugroupid


Sample Request /user/

Sample Request Body {usergroupid: 29bc6362-22d5-43e7-906a-88df8c65e212}

Sample Response Body {uuid: d851c459-9e97-4760-b929-153da7d49665 username: changeduser firstname: example lastname: user email: [email protected] role: admin uuid: 0550e896-8b93-4aef-82f1-7548b6dfcf0c username: devuser firstname: developer lastname: test email: [email protected] role: developer }

Table 37 Get User by ID API

Description GET User by ID

Endpoint /user/:uuid

HTTP Method GET

Note - Query Parameters ugroupid, uuid


Sample Request /user/:0550e896-8b93-4aef-82f1-7548b6dfcf0c


54

Sample Request Body {usergroupid: 29bc6362-22d5-43e7-906a-88df8c65e212 Uuid: 0550e896-8b93-4aef-82f1-7548b6dfcf0c }

Sample Response Body {uuid: 0550e896-8b93-4aef-82f1-7548b6dfcf0c

username: devuser firstname: developer lastname: test email: [email protected] role: developer}

Table 38 Enable Registered user API

Description ENABLE an User Endpoint /user/username/enable

HTTP Method PUT

Note -


Request/Response Format uuid/-

Sample Request /user/devuser/enable

Sample Request Body {uuid: 0550e896-8b93-4aef-82f1-7548b6dfcf0c username: devuser}


Table 39 Disable User API

Description DISABLE an User

Endpoint /user/username/disable/

HTTP Method PUT

Note


Request/Response Format uuid/-

Sample Request /user/devuser/disable

Sample Request Body {uuid: 0550e896-8b93-4aef-82f1-7548b6dfcf0c username: devuser}


Table 40 Create new Usergrup

Description CREATE a new Usergroup

Endpoint /usergroups/:ugroupid/ HTTP Method POST

Note Password will be transferred and saved encrypted

Query Parameters -

Request/Response Format json/json Sample Request /usergroups/register/

Sample Request Body {name: SMEorganizationname


55

password: SMEPassword} Sample Response Body {name: SMEorganizationname

ugroupid: 29bc6362-22d5-43e7-906a-88df8c65e212}

Table 41 Register user API

Description REGISTER a new User

Endpoint /user/register HTTP Method POST

Note uuid is generated automatically, password is transferred and saved encrypted

Query Parameters - Request/Response Format json/json

Sample Request /user/register/

Sample Request Body {username: exampleuser firstname: example lastname: user email: [email protected] company: SME}

Sample Response Body {uuid: d851c459-9e97-4760-b929-153da7d49665 username: exampleuser firstname: example lastname: user email: [email protected] company: SME}

Table 42 Login API

Description LOGIN a User Endpoint /user/login/

HTTP Method POST

Note Transfer of password is secured

Query Parameters -


Sample Request /user/login

Sample Request Body {username: userexampleuser password: adminpwd}

Sample Response Body {uuid: d851c459-9e97-4760-b929-153da7d49665 username: exampleuser firstname: example lastname: user email: [email protected] company: SME role: admin password: adminpwd usergroup: SMEorganizationnam }





56

Table 43 Update Credentials API

Description UPDATE user password

Endpoint /user/password/

HTTP Method PUT

Note -

Query Parameters -

Request/Response Format json/- Sample Request /user/password

Sample Request Body {uuid: d851c459-9e97-4760-b929-153da7d49665 username: userexampleuser password: adminpwd}


5.2 Unicorn Development Perspective The Unicorn IDE Plugin realizes the Development Perspective of the Unicorn Dashboard Layer and it is built on top of the popular open-source and extensible cloud IDE, Eclipse Che. One of the main Unicorn incentives is to empower and promote DevOps culture within IT teams in SMEs. To achieve this, the Unicorn IDE plugin benefits from the following features of Che:

• Portability. No installation or system dependencies are needed to run the IDE. The IDE is accessible through any modern web browser that has stable connection to the internet.

• Collaborative environment. Developers can work in teams and have access to various workspaces and code artifacts based on the team they belong to.

• Embedded CI/CD pipeline.

• Containerized runtime environments. Developers can define and customize their runtime environments which are enclosed within Docker containers coupled with data persistence mechanisms, environment variables, projects and commands which are associated with them, as well as resources allocation attributes.

The other main incentive of Unicorn is to allow developers create Unicorn-enabled microservices, that are Docker containerized microservices enhanced with elasticity, security, and privacy features, with configurable container and application metrics and risk and vulnerability assessment capabilities. Through the Unicorn Che IDE Plugin, developers can enhance their applications with the aforementioned Unicorn policies and capabilities by adding annotations on their source code which are provided by the Unicorn Design Libraries as described in Chapter 3 of this document. Prior to further elaborating on implementation and configuration aspects of the Unicorn IDE Plugin, it is a good opportunity to revisit some useful Eclipse Che terminology, which was sufficiently presented on Unicorn Deliverable D1.2 and will assist in understanding the nature of the Unicorn IDE Plugin. A workspace is defined to be an isolated development space that contains the project code files and all of their dependencies necessary to edit, build, run, and debug them. A workspace contains one or more runtimes in the form of Docker containers that can be dynamically configured to host the running environment of a Che project. Figure 24 showcases how these terms are being utilized by Eclipse Che.


57

Figure 24 Eclipse Che Architecture

In order to address the aforementioned goals and offer a concrete development environment for microservices with the features described above, Unicorn extended the functionality of existing Eclipse Che workspace and assemblies in order to include support for the following (Figure 25):

• The Unicorn Design Libraries that are pre-compiled within the Unicorn Che Workspace and shipped to the developers whenever the latter requests to use the Unicorn IDE Plugin in order to develop Unicorn enabled.

• The Unicorn Runtime environment which is a containerized environment that includes all necessary components, that can interpret annotations and contain the service graph of the deployed application and an instance of the applied policies design-time and runtime policies.

Figure 25 Unicorn Che IDE Plugin High-Level Architecture

The runtime environment of the Unicorn Che IDE plugin plays an important role for the overall development perspective. The runtime environment contains all the necessary configurations for a Unicorn enabled microservice to be properly developed, enriched, and tested at design-time. Its high importance can be identified at the fact that all four Unicorn Design Libraries are pre-compiled as jar files and shipped to the


58

developers upon access to the developer IDE through the web browser. It is built and configured as an Eclipse Che stack which is defined as a is a runtime configuration for a workspace. A stack is a JSON definition with meta information for the runtime environment which is described as a Dockerfile. Stacks are loaded from a JSON file that is packaged into resources of a special component deployed with a workspace master, only when the database is initialized, i.e. when a user first stats Che. Figure 26 depicts the JSON configuration of the Unicorn Che stack. As the figure shows the development runtime environment consists of a Unicorn container which was described in Section 4.4. The main command that runs the application is mvn spring-boot:run, which is a command that instructs the maven spring plugin to bootstrap the Spring application. Launching the application through Spring-Boot is due to the choice made for the early release of Unicorn to support microservices developed using the Spring Framework.


59

Figure 26 Unicorn Stack Configuration for Che


60

6 Conclusion In this deliverable, we presented all the implementation aspects and major decisions that have been taken in order to realize a very important layer of the overall Unicorn deployment at its early release. This layer includes Unicorn Design Libraries, the IDE Plugin and aspects regarding the packaging of Unicorn enhanced microservices within Docker containers. To begin with, we presented the major challenges of the current state-of-the-art technologies and the most recent advances in key technology axes that assist Unicorn to make significant progress in reaching its goals. These technologies are divided in three main pillars. The first pillar refers to the representation of Unicorn applications as enhanced service graphs. It has been pointed out that a service graph can be described as a set of nodes and edges and Unicorn policies can be added as properties on them. To this end, OASIS Tosca offers a specification and a language that utilizes the concepts of Topology Templates that consist of Node, Relationship and Policy Templates such that can be easily transformed and represent, in a formal way, a Unicorn service graph. The second pillar deals with the container environment and more specifically with Docker Technology stack. Docker as the most widely used container engine has been selected as the container runtime environment for Unicorn microservices. It can be extended and enriched with additional functionality that range from service composition through the Docker Compose to container orchestration using popular backends such as Swarm, Kubernetes or Mesos. The last pillar widens modern perception regarding the various container architectural patterns with the most significant being the Sidecar Architecture pattern which is implemented by popular tools such as Google’s Istio and Envoy Proxy. One of the main outcomes of the WP2 is the specification and definition of a formal model for the design libraries. In Unicorn there are four design libraries that are used by developers to enrich their application with functionality and policies that will allow their application to do the following: i) provide smart auto-scaling functionality that will be triggered based on user-defined policies and strategies and collected metrics on the application and infrastructure level, ii) perimeter security functionality that will alert users for security violations regarding incoming and outgoing traffic, iii) privacy and authorization functionality that will restrict data access and movement and iv) risk and vulnerability assessment that will scan user developed application for potential vulnerabilities. All these functionalities can be either offered either as annotations that will decorate code at design-time or can be applied through the policies editor of the management perspective at runtime as well. It should be noted that all design libraries are described using the EBNF formal language at the early release only Java annotations are supported. Finally, the Unicorn Dashboard and the container execution environment structure is being described. As already mentioned, the Dashboard consists of two perspectives, one for development and one for management each addressing the needs of different user actors of the system. Through the management perspective administrators and product managers are able to define runtime policies, monitor the performance of their deployments, manage containers lifecycle and compose applications through the service graph. All communications at the management perspective with the underlying Unicorn platform takes place through the publicly exposed APIs of the components of the platform. At the development perspective, developers are able to use the aforementioned design libraries in order to develop Unicorn enabled microservices using the offered annotations. All developed microservices can then be deployed to available cloud offerings through the Dashboard. However, before the deployment step, microservices are packages within special Unicorn Docker container images that are pre-bundled with all Unicorn execution agents that implement the functionality offered by the design libraries at the execution layer.


61

7 References [1] Eclipse Che Cloud IDE, “http://www.eclipse.org/che/.” . [2] Unicorn, “Unicorn Deliverable D1.1 Stakeholders Requirements Analysis.” 2017. [3] Unicorn, “Unicorn Deliverable D1.2 Unicorn Reference Architecture,” 2017. [4] Martin Fowler, “Microservices a definition of this new architectural term.” [Online]. Available:

https://martinfowler.com/articles/microservices.html. [5] Docker Inc., “Docker Compose.” . [6] Docker, “What is Docker?” [Online]. Available: https://www.docker.com/what-docker. [7] Snort, “https://www.snort.org.” . [8] Envoy Proxy, “https://www.envoyproxy.io.” . [9] OASIS TOSCA Committee, “OASIS Topology and Orchestration Specification for Cloud Applications

(TOSCA).” . [10] D. Engine, “https://docs.docker.com/engine/.” . [11] Dockerfile, “https://docs.docker.com/engine/reference/builder/.” . [12] Docker Swarm, “https://docs.docker.com/engine/swarm/swarm-tutorial/.” . [13] Apache Mesos, “http://mesos.apache.org/.” . [14] Kubernetes, “http://kubernetes.io/.” . [15] K. Pod, “https://kubernetes.io/docs/concepts/workloads/pods/pod/.” . [16] B. Burns and David Oppenheimer, “Design patterns for container-based distributed systems.” [17] Istio, “https://istio.io/.” [18] Eclipse Modeling Framework, “https://www.eclipse.org/modeling/emf/.” . [19] Xtext, “https://www.eclipse.org/Xtext/.” [20] Unicorn, “Unicorn Deliverable D4.1 Continuous Security and Privacy-by-Design Mechanisms – Early

Release.” [21] V. C. Hu et al., “Guide to Attribute Based Access Control (ABAC) Definition and Considerations,” 2014. [22] D. F. Ferraiolo, R. Chandramouli, V. C. Hu, and D. R. R. Kuhn, “A Comparison of Attribute Based Access

Control (ABAC) Standards for Data Service Applications,” NIST Spec. Publ. 800-178, 2016. [23] OASIS, “OASIS eXtensible Access Control Markup Language (XACML) TC.” [24] Apache Maven, “https://maven.apache.org.” . [25] Java Community Process, “https://www.jcp.org/en/jsr/detail?id=250.” . [26] Java Community Process, “https://www.jcp.org/en/jsr/detail?id=269.” . [27] Java Community Process, “https://www.jcp.org/en/jsr/detail?id=308.” . [28] Eclipse AspectJ, “https://eclipse.org/aspectj/.” . [29] U. Xenial, “http://releases.ubuntu.com/16.04/.” . [30] Alpine Linux, “https://alpinelinux.org.” [31] Dockerhub, “https://hub.docker.com.” . [32] Thymleaf, “https://www.thymeleaf.org.” . [33] Angular JS, “https://angular.io.” . [34] Spring MVC, “https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html.” . [35] Spring Boot, “https://projects.spring.io/spring-boot/.” . [36] Spring Security, “https://docs.spring.io/spring-

security/site/docs/5.0.0.RELEASE/reference/htmlsingle/.” .


62

8 Annex

8.1 Unicorn Enhanced Docker-Compose Model Specification

8.1.1 The format and the specification of the Unicorn Docker-Compose model

version: "versionNum" services: serviceName1: image: imageName1:tag expose: - "exposedPortNumber1" - "exposedPortNumber2" networks: - networkName ports: - "hostPort1:containerPort1" - "hostPort2:containerPort2" volumes: - hostVolumePath1:containerVolumePath1 - hostVolumePath1:containerVolumePath2 configs: - my_config - my_other_config secrets: - my_secret - my_other_secret restart: always | on-failure | unless-stopped otherService: build: context: . dockerfile: Something labels: label1: "labelValue" label2: "" args: arg1: argValue arg2: cache_from: - alpine:latest - corp/web_app:3.14 cap_add: - capabilityName1 cap_drop: - capabilityName2 - capabilityName3 command: ["command", "arg1", "arg2", "arg3"] configs:


63

- source: my_config target: containerPath uid: 'uidNumber' gid: 'gidNumber' mode: modeNumber container_name: my-container-name devices: - "hostDevicePath1:containerDevicePath1" - "hostDevicePath2:containerDevicePath2" depends_on: - serviceName1 - serviceName2 dns: - ip1 - ip2 dns_search: - dnsDomain1 - dnsDomain2 tmpfs: - /tmpfsPath1 - /tmpfsPath2 entrypoint: - entrypoint - arg1 - arg2 - arg3 env_file: - envFilePath1.env - envFilePath1.env environment: - VarName1=varValue1 - VarName2=varValue2 - VarNam3 expose: - "portNumber1" - "portNumber2" external_links: - containerName1 - project_1:serviceName5 - project_1:serviceName6 extra_hosts: - "hostName1:ip1" - "hostName3:ip3" healthcheck: test: ["CMD | CMD-SHELL | NONE", "command", "arg1", "arg2"] interval: 1m30s timeout: 10s retries: 3 start_period: 40s labels:


64

label3: "labelValue3" label4: "" logging: driver: "none | syslog | json-file" options: syslog-address: "protocol://ip" max-size: "200k" max-file: "10" networks: networkName2: aliases: - alias1 - alias3 networkName3: aliases: - alias2 ports: - target: 80 published: 80 protocol: tcp mode: host - target: 90 published: 90 protocol: udp mode: ingress secrets: - source: my_secret target: containerSecrePath uid: 'uidNumber' gid: 'gidNumber' mode: modeNumber stop_grace_period: 1m30s stop_signal: SIGUSR1 sysctls: kernelParam1: value1 kernelParam2: value2 ulimits: nproc: number nofile: soft: number hard: number volumes: - type: volume source: myVolumeData target: /myVolumeData read_only: boolean volume: nocopy: boolean - type: bind source: myBindData


65

target: /myBindData bind: propagation: mode - type: tmpfs source: myTmpfsData target: /myTmpfsData tmpfs: size: number restart: "no" volumes: volumeName1: external: boolean driver: driverName driver_opts: opt1: "value1" opt2: 2 labels: label1: "value1" label2: "" name: my-app-data volumeName2: external: name: actual-name-of-volume networks: envoymesh: network1: driver: driverName driver_opts: opt1: "value1" opt2: 2 attachable: boolean ipam: driver: driverName config: - subnet: IP/CIDR internal: boolean labels: label1: "value1" label2: "" external: boolean name: my-app-network network2: external: name: myExternalNet configs: my_config:


66

file: ./my_config.txt my_other_config: external: true secrets: my_other_secret: external: true my_secret: file: ./my_secret.txt x-balance: - prefix: "prefixPath1" weighted_clusters: - name: "serviceName1" weight: percentage_0-100 - name: "serviceName2" weight: percentage_0-100 - prefix: "prefixPath2" weighted_clusters: - name: "serviceName1" weight: percentage_0-100 - name: "serserviceName12" weight: percentage_0-100 x-security: snort: - cluster: "serviceName" Flavor: - "FlavorName" Id: - "SecurityConfigId" Intrusion Handling Type: - "IDS | IPS" Response: - "report to administrator | fire automated rule | report plus fire rule" vulnerability: - cluster: "serviceName" Access Mode: - "NORMAL | ADMIN | BOTH" Accuracy Level: - "LOW | MEDIUM | HIGH" Frequency Unit: - "HOURS | DAYS | WEEKS | MONTHS | YEARS" Frequency Value: - int Id: - "RiskAssessmentConfigId" Mode: - "INTERNAL | EXTERNAL | BOTH"


67

Performance Level: - "LOW | MEDIUM | HIGH" Response: - "report to administrator | fire automated rule | report plus fire rule" x-elasticity: - name: "name" trigger: - info: "triggerInformation" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: int - info: type: “COUNT_RUNNING” | “COUNT_UNHEALTHY” |“HOURLY_COST” members: - resource: "serviceName" clusters: - "zoneName" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: double - info: type: COUNT_RUNNING members: - resource: "serviceName" clusters: - "zoneName" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: int action: scaleOut: - count: int resource: "serviceName" cluster: "zoneName" cooldown: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” warmup: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” priority: int - name: "name" trigger: - info: "triggerInformation" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: int - info: type: “COUNT_RUNNING” | “COUNT_UNHEALTHY” |“HOURLY_COST” members: - resource: "serviceName" clusters:


68

- "zoneName" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: double - info: type: COUNT_RUNNING members: - resource: "serviceName" clusters: - "zoneName" relOp: “LT” | “GT” | “EQ” | “NEQ” | “GTE” | “LTE” value: int action: scaleOut: - count: int resource: "serviceName" cluster: "zoneName" cooldown: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” warmup: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” priority: int x-insights: - id: “insightID” timeWindow: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” composition: groupFunction: “ARITHMETIC_MEAN” | “GEOMETRIC_MEAN” | “HARMONIC_MEAN” | “MEDIAN” | “P95” | “p99” | “COUNT” | “SUM” | “MIN” | “MAX” | “VARIANCE” metric: “metricName” members: tags: - “tag” - id: “insightID” timeWindow: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours” composition: groupFunction: “ARITHMETIC_MEAN” | “GEOMETRIC_MEAN” | “HARMONIC_MEAN” | “MEDIAN” | “P95” | “p99” | “COUNT” | “SUM” | “MIN” | “MAX” | “VARIANCE” metric: “metricName” members: tags: - “tag” operation: “+” | “-” | “*” | “/” | “DIV” | “MOD”


70

max: int | String initValue: int | String collectionEvent: duration: int unit: “Milliseconds” | “Seconds”| “Minutes”| “Hours”

8.1.2 Unicorn Compose File Extended Example

version: "2" services: front-envoy: build: context: . dockerfile: Dockerfile-frontenvoy expose: - "80" - "8001" networks: - envoymesh ports: - "8000:80" - "8001:8001" volumes: - ./front-envoy.json:/etc/front-envoy.json service2: build: context: . dockerfile: Dockerfile-service environment: - SERVICE_NAME=2 expose: - "80" networks: envoymesh: aliases: - service2 volumes: - ./service-envoy.json:/etc/service-envoy.json service1: build: context: . dockerfile: Dockerfile-service environment: - SERVICE_NAME=1 expose:


71

- "80" networks: envoymesh: aliases: - service1 volumes: - ./service-envoy.json:/etc/service-envoy.json networks: envoymesh: x-balance: - prefix: "/trace" weighted_clusters: - name: "service1" weight: 90 - name: "service2" weight: 10 - prefix: "/hello" weighted_clusters: - name: "service1" weight: 50 - name: "service2" weight: 50 x-security: snort: -cluster: service1 Flavor: -“speed+accuracy” Id: -“WebAppPerimeterSecConfig” Intrusion Handling Type: -“IDS” Response: -“report to administrator” vulnerability: -cluster: service1 Access Mode: -“NORMAL” Accuracy Level: -“HIGH” Frequency Unit: -“WEEKS” Frequency Value: -“1.0” Id: -“WebAppRiskAssessConfig” Mode:


72

-“INTERNAL” Performance Level: -“HIGH” Response: -“report to administrator” x-elasticity: - name: svc-streaming_scale-out_cpu trigger: - info: mean_svc-streaming_cpu_user_utilization relOp: GTE value: 60 - info: type: HOURLY_COST members: - resource: svc-streaming clusters: - eu-zone-1 relOp: LT value: 1.5 - info: type: COUNT_RUNNING members: - resource: svc-streaming clusters: - eu-zone-1 relOp: LT value: 10 action: scaleOut: - count: 1 resource: svc-streaming cluster: eu-zone-1 cooldown: duration: 5 unit: MINUTES warmup: duration: 5 unit: MINUTES priority: 1 - name: svc-streaming_scale-in_cpu trigger: - info: mean_svc-streaming_cpu_idle relOp: GTE value: 80 - info: type: COUNT_RUNNING members:


73

- resource: svc-streaming clusters: - eu-zone-1 relOp: GT value: 2 action: scaleIn: - count: 1 resource: svc-streaming cluster: eu-zone-1 cooldown: duration: 5 unit: MINUTES priority: 1 x-insights: - id: mean_svc-streaming_cpu_user_utilization timeWindow: duration: 5 unit: MINUTES composition: groupFunction: ARITHMETIC_MEAN metric: cpu_user_utilization members: tags: - svc-streaming - id: mean_svc-streaming_cpu_idle timeWindow: duration: 5 unit: MINUTES composition: lhside: lhside: 100 operation: "-" rhside: groupFunction: ARITHMETIC_MEAN metric: cpu_user_utilization members: tags: - svc-streaming operation: "-" rhside: groupFunction: ARITHMETIC_MEAN metric: cpu_sys_utilization members: tags: - svc-streaming x-monitoring: - id: service-1


74

tags: - svc-streaming probes: - id: cpu_probe metrics: - name: cpu_user_utilization description: CPU utilization by the user group: computational type: FLOAT unit: "%" min: 0 max: 100 initValue: 0 - name: cpu_sys_utilization description: CPU utilization by the system group: computational type: FLOAT unit: "%" min: 0 max: 100 initValue: 0 collectionEvent: duration: 5 unit: SECONDS - id: mem_probe metrics: - name: mem_used description: Amount of memory used in KB group: memory type: FLOAT unit: KB min: 0 max: infinity initValue: 0 - name: mem_cached description: Amount of KB in cache group: memory type: FLOAT unit: KB min: 0 max: infinity initValue: 0 collectionEvent: duration: 60 unit: SECONDS - id: service-2 tags: - svc-streaming probes: - id: cpu_probe


75

metrics: - name: cpu_user_utilization description: CPU utilization by the user group: computational type: FLOAT unit: "%" min: 0 max: 100 initValue: 0 - name: cpu_sys_utilization description: CPU utilization by the system group: computational type: FLOAT unit: "%" min: 0 max: 100 initValue: 0 collectionEvent: duration: 5 unit: SECONDS - id: mem_probe metrics: - name: mem_used description: Amount of memory used in KB group: memory type: FLOAT unit: KB min: 0 max: infinity initValue: 0 - name: mem_cached description: Amount of KB in cache group: memory type: FLOAT unit: KB min: 0 max: infinity initValue: 0 collectionEvent: duration: 60 unit: SECONDS

Unicorn Libraries, IDE Plugin, Container Packaging and...

Documents

Transcript of Unicorn Libraries, IDE Plugin, Container Packaging and...