Automated Orchestra for Industrial Automation680151/FULLTEXT01.pdfChapter 1 is Introduction and will...
53
Master Thesis in Intelligent Embedded Systems 15Credit advanced level School of Innovation, design and Engineering Automated Orchestra for Industrial Automation On Virtualized Multicore Environment Author: Nesredin M.Mahmud
Transcript of Automated Orchestra for Industrial Automation680151/FULLTEXT01.pdfChapter 1 is Introduction and will...
Master Thesis in Intelligent Embedded Systems
15Credit advanced level
School of Innovation, design and Engineering
Automated Orchestra for Industrial Automation
On
Virtualized Multicore Environment
Author: Nesredin M.Mahmud
ii
ABSTRACT
Industrial control systems are applied in many areas e.g., motion control for industrial
robotics, process control of large plants such as in the area of oil and gas, and in large
national power grids. Since the last decade with advancement and adoption of virtualization
and multicore technology (e.g., Virtual Monitoring Machine, cloud computing, server
virtualization, application virtualization), IT systems, automation industries have benefited
from low investment, effective system management and high service availability. However,
virtualization and multicore technologies have posed a serious challenge to real-time
systems, which is violating timeliness and predictability of real-time application running on
control systems. To address the challenge, we have extended a real-time component-based
framework with virtual nodes; and evaluated the framework in the context of virtualized
multicore environment. The evaluation is demonstrated by modeling and implementing an
orchestra application with QoS for CPU, memory and network bandwidth. The orchestra
application is a real-time and distributed application deployed on virtualized multicore PCs
connected with speakers. The result shows undistorted orchestra performance played
through speakers connected to physical computer nodes. The contribution of the thesis can
be considered: 1) extending a real-time component-based framework, Future Automation
Software Architecture (FASA) with virtual nodes using Virtual Computation Resource (VCR)
and 2) design and installation of reusable test environment for development, debugging and
testing of real-time application on a network of virtualized multicore environment.
Date: 13 September 2013
Carried out at: ABB Corporate Research, Västerås
Advisor at MDH: Moris Behnam
Advisor at ABB Corporate Research: Kristian Sandström
Examiner: Thomas Nolte
iii
Acknowledgement
I would like to thank my supervisor Kristian Sandström and examiner professor Thomas Nolte for giving me this interesting thesis opportunity. I would like to appreciate Kristian Sandström once more and Aneta Vulgarakis for their invaluable guidance and assistance during my thesis work. Veronika, I admire your willingness to help me solve technical problems.
Furthermore, I would like to express my deep gratitude to my supervisor Moris Behnam
for his encouragement and loving approach; and also taking the time to read my report.
Special thanks to my examiner Thomas Nolte for understanding my enthusiasm to do the
work and recommend me to this position. I would like to thank my family for cherishing me
moral support. Finally, thank you to ABB and colleagues – it was fun and exciting!
iv
NOMENCLATURE
Abbreviation
ICS Industrial Control System
FASA Future Automation Software Architecture
DCS Distributed Control Systems
RTOS Real-Time Operating System
VCR Virtual Computational Resource
VCPU Virtual CPU
VMM Virtual Machine Monitor
IEC International Electrotechnical Commission
HVM Hardware-assisted Virtual Machine
PV ParaVirtualization
Dom0 Domain 0 – to show guest machine runs at privilege mode (ring 0)
DomU Domain U – to show guest machine runs at unprivileged mode
SDN Software-defined Networking
SDLC Software Development Life Cycle
IOMMU Input/Output Memory Management Unit
PVHVM ParaVirtualized HVM
PVOPS ParaVirualized Options (paravirt-ops)
4DIAC Framework for Distributed Industrial Automation and Control
BVT Borrowed Virtual Time
BW Bandwidth
pCPU Physical CPU
vCPU Virtual CPU
SEDF Simple Earlier Deadline First
VNC Virtual Network Computing
SPICE Simple Protocol for Independent Computing Environment
PECOS PErvasive Component Systems
VIF Virtual InterFace
TCF Target Communication Framework
SSH Secure Shell
v
CONTENTS
Chapter 1 INTRODUCTION 7
1.1 Introduction ........................................................................................................ 7
1.2 Related work ....................................................................................................... 8
1.3 Problem formulation .......................................................................................... 9
1.4 The contribution of the thesis is two folds ........................................................ 10
Extending FASA real-time component-based framework to virtual nodes ......................... 10 Design Reusable Test Environment ....................................................................................... 10
1.5 Analysis of the problem ..................................................................................... 11
Setting up of Virtualized Multicore Environment .................................................................. 11 Development of Automated Orchestra Application .............................................................. 12
Chapter 2 BACKGROUND 14
2.1 Theoretical Background .................................................................................... 14
Component-Based Real-time Software Development (CBSD) ............................................. 14 Future Automation Software Architecture (FASA) ............................................................... 15 Virtualization .......................................................................................................................... 17
2.2 Technological Background ................................................................................ 18
Xen Hypervisor (Xen pronounced /’zɛn/) ............................................................................. 18 Open vSwitch .......................................................................................................................... 19 4DIAC-IDE .............................................................................................................................. 20
Chapter 3 Development of Orchestra components 21
3.1 Component development mechanism .............................................................. 21
3.2 Orchestra organization ..................................................................................... 22
3.3 Define requirements ......................................................................................... 22
3.4 Distributed orchestra application overview ..................................................... 25
3.5 Component modelling ...................................................................................... 26
3.6 Template generated by 4diac-fasa plug-in ....................................................... 27
Application directory .............................................................................................................. 27 Library directory ..................................................................................................................... 27 Module directory..................................................................................................................... 28
3.7 Standard Midi file format and partitioning ...................................................... 28
Standard MIDI file format ..................................................................................................... 28 Software tools and libraries .................................................................................................... 29 Midi file partitioning and rhythm message ........................................................................... 30
3.8 Algorithm Implementation for functional blocks ............................................. 31
Conductor Functional Blocks implementation...................................................................... 31 Principal Functional Block implementation .......................................................................... 31 Musician Functional Block implementation ......................................................................... 32
Chapter 4 Deployment of Orchestra Application 33
4.1 Setting up - Network of Virtualized Multicore environment ............................ 33
Host installation: Ubuntu-12.04 LTS .................................................................................... 33 Xen hypervisor installation .................................................................................................... 34 Open vSwitch installation ....................................................................................................... 34 VM installation: yocto VM, Ubuntu-12.04 LTS headless ...................................................... 34
4.2 Define Virtual computation resource ............................................................... 36
VCR with QoS for CPU, memory and network BW ............................................................... 36 Application Schedule and Execution ..................................................................................... 38
vi
4.3 Orchestra Application Management ................................................................. 39
FASA component manager ..................................................................................................... 39 Libvirt and Virt-manager ....................................................................................................... 39 Orchestra distribute management script ............................................................................... 39
Chapter 5 CONCLUSION 40
5.1 Result ................................................................................................................ 40
5.2 Future work ....................................................................................................... 41
Extending orchestra application and development............................................................... 41 Defining virtual computational resource ............................................................................... 41
Chapter 6 REFERENCES 42
Chapter 7 Appendix 1
A. Orchestra modeling using 4DIAC ........................................................................1
B. Orchestra component resource allocation on a single host and scheduling ...... 4
C. Orchestra component resource allocation on a network level and scheduling .. 5
D. Virtual machine configuration xen hyprevisor ................................................... 7
E. HP Compaque Elite 8300 Ultra-slim Desktop summary ................................... 8
F. Open vSwitch throughput test result using iperf tool..........................................1
G. Open vSwitch – jitter and latency test .................................................................1
7
Chapter 1
INTRODUCTION
1.1 Introduction
Industrial control systems (ICS) are found in many areas, e.g., in motion control for
industrial robotics, process control of large plants such as in the area of oil and gas, and in
large national power grids. A control system controls processes using control application by
interacting with environment through sensors and actuators. ICS are becoming large and
complex now days due to many reasons. For instance, Distributed Control Systems, DCS such
as ABB 800xA automation provides extended services beyond controlling simple industrial
processes, such as energy efficiency and asset management, energy savings and operator
effectiveness [1]. Likewise, embedded control systems are imposed with more regulatory
requirements than before for safety reasons (such as IEC 61508, ISO 26262) and other sub
segment standards in energy, process automation and control. As a result, it is common and
desirable for a single industrial control system to execute more than one application
simultaneously to preserve physical space and ease of management.
To meet current industry demands for large and complex software systems, integration
of software technologies and consolidation of hardware and software is crucial; for example
adoption of virtualization and multicore technology. Virtualization powered by hypervisors
such VMware ESX and Xen provides the ability to run multiple virtual machines in a single
physical node by abstracting underlying hardware resources such as CPU, memory and
communication systems; hence, low hardware investment and high energy cost savings. Even
service availability and maintenance become easier due to compartment of virtual machines
efficiently on a single physical node. Similarly, multicore technology is benefiting general
computer industry at large and now a day being applied to industrial and embedded systems
such as in mobile systems. Multicore processors have multiple cores in a single chip;
therefore, it enables multiple applications to run parallel in each core without the need for
separate hardware systems while maintaining application integrity as compared to unicore
systems. Moreover consolidation of virtualization and multicore technology can provide
spatial and temporal isolation of applications running in a single machine.
Figure 1: Virtualized multicore platform
8
For example in Figure 1, multiple virtual machines are running on top of multicore
platform which is virtualized using Xen hypervisor. The hypervisor allocates one-to-one
mapping of cores to virtual machines to achieve high degree of temporal and spatial isolation
of applications running on each virtual machines.
The benefit of virtualization and multicore technology is a trade-off with challenges they
incur in real-time systems. Due to complexity of multicore architecture, e.g., shared bus,
shared memory and multi-threading, porting of real-time application across different
hardware platforms has been challenging. Likewise in virtualization technology, hypervisor
schedulers such as Xen credit scheduler incur significant over head for example as compared
to traditional real-time schedulers. And this introduces jitter to virtual machine execution
which leads to unpredictable real-time application execution. Real-time component-based
platforms such as FASA [2], CAmkEs [3], PECOS [4] provide development and runtime
environment for real-time component-based application. And the purpose of the thesis is to
evaluate real-time component-based framework, FASA in the context of virtualized multicore
environment. And this will be demonstrated using an automated orchestra application. The
orchestra application is a real-time and distributed application developed using real-time
component based software development platform. Finally, the application is expected to
produce acoustically pleasant orchestra (unnoticeable delay and latency) through speakers
connected to each physical computer node.
The rest of the report is organized in the following way. Chapter 1 is Introduction and
will continue to analyze the thesis problems in depth. Chapter 2 is Background to theoretical
and technological aspect of real-time component based software development, virtualization
and software technologies used in the thesis scope of work. Chapter 3 is development of
orchestra application; and chapter 4, deployment of orchestra application on network of
virtualized multicore PCs. Finally, chapter 5, result, conclusion and future work is discussed.
1.2 Related work
In this thesis work, using orchestra to evaluate real time system is motivated by Julien et
al. [5]. Among others they used orchestra to show synchronization and integration of several
IEC 61499 standard compliant functional blocks on a network of NetBurner MOD5272-
100CR systems. They were able to show how the orchestra was played without distortion
using functional blocks showing applicability of IEC 61499 standard for soft real-time
application. In this thesis research, however, we will use similar automated orchestra
application on virtual and multicore environment. In addition, we will evaluate a real-time
component framework as compared to FORTE or similar runtime environment used in the
above mentioned research.
There have been many attempts to make Xen hypervisor applicable for real-time
application, such as in the case of RT-Xen [6] [7]. RT-Xen implements four server based
hierarchical scheduling algorithms (periodic, polling, sporadic and Deferrable server) which
are based on fixed-priority preemptive scheduling. The algorithms were evaluated and
compared to Xen default Credit scheduler for overhead measurements (latency and context-
switches) over 10 seconds of execution; and the result showed an average of 0.2% time loss
for RT-Xen schedulers and 0.04% for Xen Credit scheduler. To minimize overhead of Xen
scheduler, we avoided using RT-Xen; instead, we mapped each virtual machine (aka
domains) to a single core so that excepting credit scheduler from load balancing VCPU across
physical cores, therefore, increasing timeliness and predictability of component execution on
domains.
9
Little work is done, at least to my knowledge, on evaluation of timeliness and
predictability of real-time component-based framework on virtual environments.
Evaluations of several component technologies for embedded system can be found in Anders
Möller et al. [8]. They used technical and development process requirements to prioritize
component technologies. Among which PECOS was found to perform better in average;
especially for resource constrained application. In this approach; however, only technical
evaluation of timeliness and predictability of FASA component model framework will be
performed for real-time and distributed application.
1.3 Problem formulation
The purpose of the thesis is to extend real-time component-based framework, Future
Automation Software Architecture, FASA with virtual nodes using Virtual Computational
Resource (VCR); and evaluate timeliness and predictability f real-time application in the
context of virtualized multicore network of PCs. And this is demonstrated by modeling and
implementing orchestra application. The application is soft real-time1 system and distributed
application which implements functionality and synchronization of orchestra performance.
The thesis work will try to address the following research sub-problem:
Problem #1: Study and implement the concept of Virtual Computational Resource,
VCR2 with QoS for CPU, memory and network bandwidth.
Underplaying computer hardware resources such as memory, CPU and network
bandwidth is divided into a set of virtual resources defined in textual format, identified in
this report as Virtual Computational Resource, which enables a virtual system to run in a host
system. Virtual Computational Resource creates a layer of abstraction to hardware resources
which makes it possible to preserve execution behavior of virtual systems across differ
hardware platforms. The benefit of VCR can easily be seen in porting real time application
from for instance from unicore to multicore system.
Problem#2: Model and implement an orchestra application on a network of virtualized
multicore PCs.
According to a research made by Nelson Posse et al. [9], human ear is capable of
recognizing latency between 20-30 milliseconds. This implies, events occurring within 20-30
milliseconds drift can be identified as separate events by our ear. Furthermore, the research
discusses the power of human perception to musical rhythm can reach to 4ms precision. This
leaves us how we can exploit our hearing and rhythm perception capability in real-time
systems. Therefore, musical orchestra will be automated to demonstrate how real-time
component-based frameworks perform under virtual computational resource.
Problem #3: Study to what degree the architecture will support software-independence
by evaluating how different allocations of application parts impact real-time behavior.
Automated Orchestra application will be deployed in one and more physical nodes; and
see how this different allocation will result in playing the orchestra properly with maintaining
timeliness and predictability of real-time application.
Problem #4: Dynamic turn ON/OFF of instruments and prepare test configurations.
1 A real-time system if missed deadline degrades performance but still is usable afterwards; for
example, out of sync an instrument produces acoustically unpleasant music but still continues to play the orchestra.
2 VCR is a computational resource obtained by partitioning an actual hardware resource e.g., CPU, memory and network bandwidth for applications as QoS.
10
1.4 The contribution of the thesis is two folds
Extending FASA real-time component-based framework to virtual nodes
The real-time component-based framework under analysis, FASA, according to previous
records does not support partitioning [2]. In this thesis work, it is extended to support virtual
nodes using Virtual Computational Resource (VCR) with QoS for Memory, CPU and network
bandwidth. VCR is a computing resource definition created by partitioning actual hardware
resource into units that can individually be used for development, allocation, verification and
analysis of parts of a system. For instance, FASA application parts, such as automated
orchestra components would be allocated to VCRs; and the QoS defined in the VCR e.g., CPU,
Memory and network bandwidth could be enforced during runtime by Virtual Machine
Monitoring (VMM) such as Xen hypervisor.
VCR can be prepared separately without the knowledge of hardware system at hand;
consequently, facilitating software engineering process e.g., requirement analysis and design.
It also preserves timeliness and predictability of real-time application parts if migrated, for
instance to a different hardware architecture than where it originally was running, e.g., from
a unicore to multi-core system. Finally, the communication performance of the test
environment as show in Figure 2 such as jitter, latency and throughput is compared over
Ethernet protocol with and without VCR.
Design and Install Reusable Test Environment
The test environment is a network of computer nodes; and consists of two nodes
virtualized using Xen hypervisor, two non-virtual nodes for benchmark purpose and a central
management system. Three virtual nodes are created per host; and are connected using
OpenvSwitch3 – an advanced virtual switch. The virtual nodes and nodes used for benchmark
are installed with Real-Time OS developed using Yocto Project Development4 approach – an
open source and collaborative project which creates custom Linux distribution for embedded
systems. Furthermore, the test environment is setup with Yocto Project Development for
System Core developers (e.g., building OS) and Yocto Application Developers (application
which run on top Yocto Machines). It provides a platform for further development,
debugging, testing and evaluation of real-time OS and network communication performance.
It has central and coordinated management for computer nodes using technologies, e.g.,
Virtual Network Computing (VNC), Target Communication Framework (TCF), Secure Shell
(SSH) and Libvirt - a library for managing platform virtualization technologies such as Xen
hypervisor, KVM,VMWare ESX. The test environment would be suitable, e.g., to develop and
cross-compile network performance tool for Yocto Machines, test high precision
synchronization of virtual nodes and Xen hierarchical scheduler development.
3 OpenvSwitch - an open source virtual switch suitable virtualization. http://openvswitch.org/ 4 Yocto Project is an open source and collaboration project for creating Linux based distribution
for embedded systems. https://www.yoctoproject.org/
Figure 2: High-level test environment
11
1.5 Analysis of the problem
We may breakdown the thesis work into two parts: Setting up of Virtualized Multicore
Environment and Development of Automated Orchestra Application. Using this approach we
will discuss how the above mentioned sub problems will be addressed throughout the thesis
work
Setting up of Virtualized Multicore Environment
Linux OS (aka GNU/Linux)5 has become the choice of academic research in OS,
mainframes, supercomputers and now days being used in embedded systems, e.g., mobile
phones, network routers, automation control. Android OS is a prominent example of Linux
OS used in embedded systems; it is build from Linux Kernel and deployed on mobile phones.
Linux OS is free and open source available in several distribution forms; example of
commercial distributions includes Ubuntu (Canonical Ltd.), Fedora (Red Hat) and
openSUSE (SUSE). Ubuntu is one of the most popular distributions in Linux community;
and it uses Debian package manager, dpkg which is powerful and easy to use. We will use
Ubuntu-12.04 as a host and management OS in setting up of virtualized multicore
environment.
The following are a brief description how virtualized multicore environment will be
established:
1. Host virtualization - A hypervisor will be used to monitor (or CPU access,
memory usage and I/O peripheral access) virtual machines. The most common
open source hypervisors at the time of writing the thesis are Xen, KVM and
VirtualBox. Xen hypervisor is chosen for I/O performance and maturity reasons.
It is supported by a number of companies such as Amazon, Cloud.com and
GoGrid; and generally performs better as compared to KVM and VirtualBox
according to some study6.
2. Virtual Machine - OS for virtual machine is based on Real-Time OS (RTOS) from
Yocto Project Development7 – a lightweight, customized Linux distribution.
3. Communication and synchronization of virtual and non-virtual nodes - Once
established the virtual environment; communication between virtual machines
is achieved through virtual switch supporting OpenFlow standard. OpenFlow
standard is an open standard which defines the control and forwarding layer of
software defined networking, SDN8; similar functions found in routing and
switching. The most common open source virtual switch supporting openflow
are open vSwitch and Indigo. Since open vSwitch hot-plug script9 is supported
by default in the latest release of xen-4.3, open vSwitch is used for
implementation. Computer nodes are synchronized to central node through
Network Time Protocol (NTP).
4. Tuning virtualization (CPU and Memory usage) - The following components of
hypervisor if tuned and configured correctly give an optimal virtualization for
execution of orchestra application.
Scheduler – Xen comes with default credit scheduler; and other
schedulers SEDF and BVT. Virtual CPU, VCP is another feature to be
considered for optimization. The more VCPU a guest machine has the
5 Linux OS: http://en.wikipedia.org/wiki/Linux 6 Xen Vs. KVM:
http://indico.cern.ch/getFile.py/access?contribId=5&resId=1&materialId=slides&confId=45282 7 Yocto project: https://www.yoctoproject.org/ 8 SDN: http://en.wikipedia.org/wiki/Software-defined_networking 9 A script that enables an auto-configuration of open vSwitch for ease of use
12
more CPU access has. In fact, to achieve predictability, Xen can be
extended with other schedulers, e.g., addition of hierarchical real-time
scheduler
I/O – Xen HVM hypervisor enables I/O device emulation and direct
device access from virtual machines. High performance is achieved when
configured with direct I/O access also called PCI pass-through in Xen.
Memory – Virtual machines should not suffer from limited memory
access. Instead, virtual machines will be allocated sufficient memory
during creation.
Once we have a working environment, we define a Virtual Computational Resource
(VCR) will QoS for CPU, memory and network bandwidth for each virtual node.
Development of Automated Orchestra Application
First a preliminary study of orchestra will be conducted such as organizational structure,
communication and synchronization of musicians. This is crucial to imitate relevant features
required for our application. Following this, orchestra application will be developed based on
approach suggested by de Almeida et al. [9] which emphasizes the need for componentization
on design and implementation level through the Software Development Life Cycle, SDLC.
Based on detail analysis of orchestra application, first will be to model and implement
components required for the application. This will be followed by deployment of the
application on virtualized multicore environment which involves resource allocation (CPU,
memory, network bandwidth) and development of application schedule.
4DIAC-IDE and FASA are the platform to be used for development and runtime
execution of orchestra application. FASA is an ABB in-house research framework for
development of distributed control applications. It is used to develop, build and run cyclic
application. Since there is close connection to development team of FASA for support, it is
favoured over other platforms such as CAmkES and PECOS for this thesis work.
The following tools will be used to develop the orchestra application:
1. FreeMind - a mind mapping tool
2. 4DIAC-IDE – an open source component-based software development IDE
based on IEC 61499 standards.
3. 4diac-fasa plug-in – a modelling transformation tool from 4IDAC to FASA
components model.
4. FASA framework - a development and runtime execution platform for
distributed control application.
Using the above listed tools, the orchestra components and distributed application will
be developed following a software development life cycle process. First, a mind mapping tool,
FreeMind is used to describe the basic picture of orchestra organization that is the conductor,
principal and musician and the hierarchical relationship. Next a UML (use case and
component diagram) tool, Umbrello is used for detail problem specification, component
design and relationship. Finally, the design will be realized using 4DIAC-IDE tool to model
and implement the prototype orchestra application.
13
Once the orchestra application is developed on the 4DIAC-IDE, a 4diac-fasa plug-in will
be used to transform the model which was previously based on IEC 61499 standards to a
FASA compatible component modelling platform for runtime execution. Since the 4DIAC-
fasa plug-in transformation does not have support yet for resource allocation devices to
components, FASA application template, which defines allocation of components, will be
manually edited. In addition, to launch the application from a central node, a script will be
developed for deploying, starting and stopping the application on the virtualized nodes.
Figure 3: orchestra application deployment logical architecture
14
Chapter 2
BACKGROUND
2.1 Theoretical Background
Component-Based Real-time Software Development (CBSD)
A component is a reusable unit of software deployment accessed through an interface10
[10]. Adopting a component based software development, especially for large and complex
software architecture simplifies development, encourages inter operability of third party
components. Since components are usually tested and maintained frequently, application
developed using component based achieves higher quality and shorter development time task
based applications. In fact, several industries have benefited from component-based
development; for instance, the global industrial automation company, ABB has implemented
component based in the new release of industrial automation and collaborative platform
product, System 800xA Extended Automation.
Industrial Control Systems have strong constraint in resource utilization, especially on
hardware resources (CPU, Memory, Network Bandwidth and Hard disk), power
consumption, and physical space. On top of this, complexities of such applications are
growing from time to time due to additional services; such services for example can be
observed on ABB System 800xA Extended Automation. The new automation platform
supports real-time system information retrieval and enables decision making right away and
has collaborative platform for effective operation [1]. In order to provide a solution to such
inevitable industrial demands, a component-based software development has to be extended
to address real-time behavior of control application. And this has been possible by
implementing a real-time component based development framework.
Real-time component-based software development addresses timeliness and
predictability demand of real-time control applications. In fact, similar functionality can be
achieved with the use of processes, semaphores, and message queues; however, their
implementation has the tendency to incur runtime overhead as compared to real-time
component due to frequent context-switches. In addition to using real-time component based
development, application need to be distributed across multiple devices; e.g., hosts, cores to
share execution load. In the following section we will discuss Future Automation Software
Architecture, FASA in detail - an example to real-time component-based software
development by ABB research group [2].
10 An Interface is with a component is a point of access point based on a well defined and accepted
standard, also called a contract.
15
Future Automation Software Architecture (FASA)
FASA [2] [11] [12] [13] is a Component-Based Real-Time Software Development
platform with its own runtime environment, deployment and management platform. It
supports two modes of application development, fast and safe mode. At compile time, the
platform enables software engineers to select either one or both mode. Fast mode enables
applications to execute in a single process; as a result, components reside in the same address
space and communicate through shared memory. Applications developed using fast mode
executes faster; however, may introduce memory corruption due to share memory. On the
other hand, safe mod allows components of the same application to reside in a separate and
dedicated address space; as a result, memory corruption is technically avoided.
Figure 4: FASA framework
FASA framework consists of FASA kernel, FASA framework manager and a schedule
policy. Each FASA kernel instance provides a FASA framework manager. The manager is
responsible for managing execution of a distributed application at runtime such as schedule
policy change and component related operation; e.g., start and stop component execution.
1. FASA Component Framework
FASA component framework comprises the following structural elements: Component,
Block, Port and Channel. A Component in FASA provides isolation of software application
parts by encompassing one or more blocks. A block is a sequence of software instruction that
defines FASA unit of execution. Data communication between blocks is achieved through
Channels. A channel in FASA is implemented as a one-to-one and unidirectional
communication between ports defined in blocks willing to transfer data. A port is a data type
interface, which is identified as input/output port, holds data for receiving and transmitting,
respectively.
Data communication between blocks is made possible through three different
mechanisms depending on components relative deployment.
- Components within the same address space – blocks within a single component
or components with the same address space use shared memory for data
communication. As compared to other communication types discussed below, it
has low communication latency due non-context switching, but high possibility
of memory corruption11 .
11 Violation of memory location integrity caused by processes interference which ultimately causes
program crash or undesirable behaviour, wiki
HOST
OS
FASA KERNEL FASA Framework ManagerSchedule Policy
Ap
pli
cati
on
FASA framework
B1 B2channel
component block portApplication
Figure 5: FASA component framework
16
Figure 6: Shared memory communication mechanism
- Components on different address space but within the same Host – FASA
implements System V message queue12 for data communication between blocks
residing on different address space. This type of communication also applies to
components located at separate cores within the same host. A sender block
sends message to a shared queue asynchronously while a receiving block
retrieves the message synchronously which means the receiving block will be
blocked until data is available. This is to avoid faulty data retrieval by the
receiving end.
Figure 7: Message queue communication mechanism
- Components on different host (physically or virtually separated hosts) –
FASA uses a network proxy to achieve data communication between blocks
residing on physically different machines. A network proxy is an application
layer protocol based on client-server model which acts on behalf of sender or
receiver end. A block ready to send data uses a Network Send Proxy; similarly,
a block ready to receive data uses Network Receive Proxy. The network proxies,
Netproxy, are special blocks in FASA. They are scheduled like a normal FASA
blocks to execute immediately after a sender block or before a receiver block.
Figure 8: Netproxy communicating mechanism
12 Message queue implementation of System V (aka SysV) which is one of the pioneer Unix OS
Sender Block Receiver Block
Shared memory
Write Read
Sender Block Receiver Block
Data areaWrite ReadData area
Message Queue
Msg_send Msg_receive
Sender Block Receiver Block
Data areaWrite ReadData area
Msg_send
Msg_receive
Network Send Proxy
Network Receive Proxy
Network
17
Virtualization
Application of virtualization dates back to 1960s when IBM mainframe was started to
host multiple applications sharing the same physical resource and running simultaneously.
Since the acquisition of x86 based machines and desktop OS such as windows and UNIX,
virtualization evolved to refer to a large scale of applications such desktop level, OS level and
application level virtualization [14]. The definition of virtualization is buzzword and is
defined differently by different authors even today; however, there is a general consensus on
the basic concept and application of virtualization. It mainly refers to logical partitioning of
physical computing resources regardless of the physical location; and by doing so operational
cost is minimized and investment is reduced. The most obvious computing resources to be
virtualized are processor, I/O device, memory, communication and storage. And
virtualization technology is used in academia and business mainly through desktop
virtualization and cloud computing [15]. It’s quite important to differentiate virtualization
from emulation. Both of them involve imitation of computing resources as though they exist
in reality; however, the methodology and purpose is different. Emulation imitates a hardware
or software of interest by translating instructions from one form to another. For example,
QEMU is an emulator; we can create or emulate a PCH HDA adapter regardless of the
underlying hardware sound adapter is, e.g., it could be Sound blaster. Virtualization uses
hypervisors to partition and manage computing resources. An emulator is usually used for
testing different processor and peripheral devices for which it is not designed for, avoiding
hardware or software expenses; and also is used in virtualization to emulate some devices to
the virtual machine.
Components of virtualization
Virtualization has two main components: a hypervisor and guest machines. A hypervisor
is a piece of software (aka called a Virtual Machine Monitor, VMM) that manages resources
such as CPU, memory and interrupts used by virtual machines. Depending on CPU mode13 a
hypervisor is classified into two types (it is not a clear cut classification though):
1. Type-1 or Bare metal hypervisor (Figure 10: Type 1 hypervisor) – executes right
on top of a hardware layer with privileged mode as compared to guest machines.
And guest machines run on top hypervisor with restricted mode of operation.
Example, Citrix XenServer, VMware ESX/ESXi.
2. Type-2 or operating system level hypervisor (Figure 9: Type 2 hypervisor)–
operates on top of conventional operating system such VMworkstation and
VirtualBox .
13 The two main CPU modes are privileged mode and restricted mode. Unless a program (a
hypervisor or OS) is in privileged mode some instructions are not allowed to execute. http://en.wikipedia.org/wiki/CPU_modes
Figure 10: Type 1
hypervisor
Figure 9: Type 2 hypervisor
18
2.2 Technological Background
Xen Hypervisor (Xen pronounced /’zɛn/)
XenHypervisor is an open source hypervisor developed by Xen Project Team14. It is a
Virtual Machine Monitoring, VMM comprising lightweight software program (<150,000
lines of code) that manages CPU, memory and interrupts for guest machines. Neither does it
manage I/O transaction nor does it manage guest machines. Instead, there is a privileged
guest machine, Dom0 which is ported with host OS usually from UNIX flavor (Linux,
FreeBSD distributions) which manages guest machines and device drivers. Xen is a type 1
hypervisor; and is base to XenCloud platform and XenARM.
The Xen architecture has the following components:
1. The Xen hypervisor – manages CPU, memory and interrupts to guest machines
2. Dom0 – a control guest machine which has driver support for guest machines
and tool stack to manage DomU.
3. DomU – a guest machine which run under restricted mode with their own OS
4. Xen Tool stack15 – an interface in command line or graphical to manage the guest
machines.
5. Modified kernel – Xen requires a modified kernel to act as a Dom0 usually found
from the mainline kernel or build from source.
Xen hypervisor supports three types of virtualization:
1. Paravirtualization (PV) – is the first type of virtualization supported by Xen. This
type of virtualization requires a modified guest machine drivers (aka backend
drivers) on the Dom0 side and a frontend drivers on the DomU side. DomU I/O
access is through Domo backend drivers. Dom0 has the actual driver to
communicate with Host hardware.
2. Hardware Virtual Machine (HVM) aka Full virtualization – this is type of
virtualization is supported on host machines with virtualization extension such
as Intel VT-x. Hardware resources such CPU, memory and I/O are emulated
14 Xen Project: http://www.xenproject.org/ 15 Xen toolstack: http://wiki.xen.org/wiki/Choice_of_Toolstacks
Xe
n c
om
po
ne
nts
Dom0 (Management
guest machine)
Xen Hypervisor Schedule MMU
toolstack
Backend driver
Qemu-Xen
PV Guest Machine
DomU
Frontend driver
HW drivers
PVHVM Guest Machine
DomU
Frontend driver
HVM guest machineDomU
Xen virtual firmware
Host HW
Figure 11: Xen components for PV, HVM and PVHVM types
19
using QEMUL device emulator. Qemu-xen16 is the Xen flavor of QEMU; it
provides a virtual firmware to DomU by giving the impression that the guest
machine is running directly on physical machine. Emulation is processor
intensive and as compared to PV, HVM performs less in I/O throughput. To solve
this problem, Xen team introduced PCI pass-through in Xen hypervisor. PCI
pass-through enables DomU to access I/O devices directly without the emulation
using Input/Output Memory Management Unit, IOMMU (Intel VT-D and AMD
IOMMU) to boost communication performance between I/O devices and DomU.
3. Paravirtualization on HVM – this is a recent feature introduced on Xen-4.3
release. It boosts HVM performance using PVHVM drivers (an optimized PV
driver for HVM); therefore, without the need for QEMU device emulation. A
kernel feature PVOPS in the host machine identifies for virtualization mode and
then divers driver request to use PVHVM drivers. Therefore, the HVM network
and disk access uses paravirtualization method (with modified driver) instead of
emulation.
Open vSwitch
Open vSwitch (open virtual switch) is a switch specially designed for bridging virtual
machines to physical network [16]. It is based on Software-defined Networking, SDN
approach which emphasizes the need for separate control and data plane17. Linux-bridge is
the default birding in Linux for virtual machines; however, it’s not designed to support multi-
server virtualization18; as a result networking in virtualization tend to go to more advanced
virtual switches such as Open vSwith and Indigo.
Openvswitch operates at user and kernel space mode. The kernel module,
openvswitch_mod.ko processes and forwards packet entering a physical network interface. If
bridging traffic to virtual machine is required or advanced switch configuration is needed the
packet is forwarded to ovs-vswitchd at user space. Initial switching configuration is stored at
ovsdb-server and using JSON/RPC, the configuration is offloaded into flowTable. After
creating a bridge, a physical Ethernet configuration is no more used.
16 Qemu-xen: http://wiki.xen.org/wiki/QEMU_Upstream 17 SDN: http://en.wikipedia.org/wiki/Software-defined_networking 18 Why openvSwitch? : http://git.openvswitch.org/cgi-
bin/gitweb.cgi?p=openvswitch;a=blob_plain;f=WHY-OVS;hb=HEAD
Op
en
vS
wti
ch c
om
po
ne
nts
Ovsdb-server
Ovs-vswitchd(slow path)
Packet classifier FlowTable
Openvswitch_mod.ko(fast path)
Kernel mode
User mode
Hash_lookup Table
Eth 0
JSON/RPC (management)
NetLink
xenbr0 xenbr1
Figure 12: open vSwitch components
20
4DIAC-IDE
Framework for Distributed Automation and Control, 4DIAC is a free, open, IEC 61499
standard compliant automation and control environment. One of the projects by the 4DIAC
initiative is 4DIAC-IDE. It is an integrated development environment of IEC 6199 compliant
automation and control, based on Eclipse framework.
An IEC 61499 standard provides the architecture, tools and rule of compliance for
development of distributed automation and systems [17]. It addresses shortcomings of its
predecessor, IEC 61131 standard e.g., programming language independence but moreover it
provides distribution and dynamic reconfiguration. The atomic execution unit of IEC 61499
standards, Functional Block, FB is the construct of an application. It has event and data
connection to interact with other FB or the environment as indicated by red and blue lines in
the 4DIAC-IDE. An application is developed by interconnecting FB instances and allocating
them a resource19. And the FBs are distributed to physical devices across a network. Events
trigger actions by executing algorithms within the FB according to Execution Control Chart,
ECC [18]. And an executed algorithm pushes data to the output I/O for transmission.
4DIAC-IDE has three main perspectives20 for developing an application based on
functional block semantics defined by IEC 61499 standards.
1. System management perspective - this is the main perspective where control
applications are modeled, devices and resources are configured
2. Type management perspective – is the perspective for managing IEC 61499
compliant types such as using pre-existed FBs, creating blocks and editing
functional blocks.
3. Deployment management perspective – finally a system configuration (resources
and devices) is prepared and application deployed on controllers launched using
runtime programs such 4DIAC - Runtime Environment, FORTE.
19 Resources can be considered processor cores, memory, bandwidth … provided by devices. 20 4DIAC-IDE Quick start:
http://sourceforge.net/apps/mediawiki/fordiac/index.php?title=4DIAC-IDE_Quickstart
Figure 13: 4DIAC-IDE perspectives
21
Chapter 3
Development of Orchestra components
The Orchestra application is a composite of components where each component
provides a service, e.g., rhythm message and timing specification to another component. The
component service is defined using functional blocks modeled using 4DIAC-IDE tool based
on IEC 61499 standards and executed on FASA runtime environment (aka FASA kernel). The
application constitutes a total of 17x components and 17x functional blocks. Software tools
employed during development are FreeMind for mind mapping, Umbrello UML modeling
tool (Use Case Model and Sequence diagram), 4DIAC-IDE and FASA for modeling and
implementing orchestra application21. In the following subsequent topics, we will discuss the
development approach, preliminary study of orchestra in music, and modeling and
implementation process: Defining requirements, modeling and implementation using the
software tools mentioned above.
3.1 Component development mechanism
Initially orchestra application components are developed and implemented using
4DIAC-IDE. Then using 4diac-fasa plug-in, the model is transformed to FASA component
model framework where execution of the application is performed. FASA component
framework does not have its own IDE; instead, the plug-in is used to transform some of the
features of 4DIAC-IDE e.g., basic functional block, sub-application of 4DIAC to blocks and
components in FASA. By the time of report writing, basic 4DIAC-IDE features such as device
creation, resource allocation and deployment is not supported by the plug-in; as a result, it is
done manually.
4DIAC-IDE is build by cloning to the source hosted on sourceforge.net. Then 4diac-fasa
plug-in is imported to the set of 4DIAC-IDE plug-ins before building it. The built IDE gives
us an eclipse-IDE with 4DIAC features and automated transformation for 4DIAC-to-FASA.
The plug-in enables an application engineer to use 4DIAC-IDE to develop application for
FASA framework which provides visualization instead of textual work on FASA; however, the
transformation is not smooth enough perhaps is under further development. The
transformation result is a set of files and directories regarding components and functional
blocks; these need to be copied manually to respected structural tree on FASA. Due to this,
for every update on the model, it is not reflected automatically on the FASA. I believe this has
to do with the deployment feature of 4DIAC not automated for FASA. The perspective is used
to deploy an application on a selected runtime environment, similar to FORTE and FBDK22,
which can be a prototype for FASA integration in the deployment perspective. We suggest
FASA deployment process be integrated in the deployment flow of 4DIAC-IDE in the future.
Before diving into the development process, it’s important to understand the context of
orchestra in music such as organizational structure, role, communication and
synchronization of musicians; and features adopted or imitated to our orchestra application.
21 Does not include resource allocation and device assignment of components, this is done using
FASA framework. 22 Functional Block Development Kit, FBDK: http://www.holobloc.com/doc/fbdk/index.htm
22
3.2 Orchestra organization
An orchestra23 is a large instrument ensemble usually led by a conductor and principals
or sub-conductors. As a rule of thumb, the conductor is responsible for leading the whole
orchestra while the principals rehearse a certain group of musicians with similar quality of
instruments such as string, percussion, woodwinds and brass. The flow command for playing
the orchestra comes from top hierarchy and goes through principal at second level. The basic
orchestra activities are imitated to suit the project intended objective that is timeliness and
predictability in multi processing environment. If the instrument is played before or after an
anticipated time (which is indicated by rhythm note), the orchestra goes out of sync. Out-of-
sync implication in this context implies to latency and jitter introduced during execution of
components in the virtualized multicore environment and transmission. According to study
[9] on human perception to such effects, many researchers share the conclusion that 20-30
ms sound delay is tolerable by our ears. What is more interesting about sound produced by
orchestra is human capability to follow rhythm even at slight deviation of 4ms at sub
conscious level. This has in an implication to the feedback we get from the orchestra
application: the feedback we get from the orchestra application need to be within 20-30ms so
that it will not be perceptible by our ears; and second even with acceptable range of latency
and jitter, the quality could vary; and this is possible to notice by following the rhythm within
high precision or around 4ms.
3.3 Define requirements
Automated orchestra application conceived from the concept of orchestra in real world
(for example, as shown from the mind map of Royal Stockholm Philharmonic Orchestra) is a
two level hierarchal of communication, which is communication between the conductor and
principals; and among principals and corresponding group of musicians. Whatever is
directed from the conductor is passed through the principals; and rehearsed by musicians
asynchronously (in our implementation the communication goal is achieved through cyclic
execution of blocks, though).
23 Orchestra wiki: http://en.wikipedia.org/wiki/Orchestra
Figure 14: Mind map of orchestra from Royal Stockholm Philharmonic Orchestra
23
The problem domain identified in the mind map is elaborated further in the Use Case Model.
Figure 15: Use Case Automated Orchestra Application
Use case name Conduct Orchestra
Participating Actors Computer Terminal User
Precondition The system is setup with roles of musicians and timing
information such as start time, tempo in a configuration file.
Basic flow 1. The user launches the application by entering
‘application run’ on the terminal.
2. Parses a configuration file and stores orchestra
organizational structure and rate (cycles/sec) at which
the orchestra to be played in a buffer.
3. Decode midi file and save it into a buffer.
4. Creates rhythm groups24 composed of multiple tracks to
be played at constant rate.
Alternative flow
Post conditions Saves the state of conducting the orchestra such as indices of the
last event25 from each rhythm group and absolute time played
since the orchestra has started.
Store the generated rhythm groups in a shared memory.
Special requirements
Use case name Conduct Musicians
Participating Actors
Precondition Rhythm groups are generated and stored in a shared memory.
Basic flow 1. Reads orchestra organizational structure from a
configuration file.
2. Split the rhythm groups into a rhythm26.
Alternative flow
24 Rhythm group is a set of rhythms. 25 Event is the occurrence of ON /OFF notes indicated in MIDI file protocol standard. 26 Rhythm is a subset of events in a single musical track/instrument in a MIDI file.
24
Post conditions Stores the rhythms in a shared memory.
Special requirements
Use case name Play Music
Participating Actors
Precondition A rhythm is generated and stored.
Basic flow 1. Parses the rhythm and identify ON/OFF events.
2. Reads delta time between subsequent events.
3. For each event, the sound system is triggered preceded
by a silence of delta time. And this will produce a sound
on the speaker.
Alternative flow
Post conditions
Special requirements A speaker should be connected to the system.
An audio system which produces a sound should be installed in
the system.
A synthesizer and sound bank that interpret the notes should
be installed in the system.
Automated orchestra application is initiated using a computer terminal. It could
deployed eit a standalone system or distributed across a network of computer nodes. The use
case, Conduct the Principals process the input files to the system and further decodes the
midi file into a group of musical note which is input to another use case, Conduct the
Musicians. The later use case distributes the musical note which is finally played in Play
Instrument use case. This process is repeated periodically until the midi file timeline is
finished playing.
Figure 16: Orchestra performance UML sequence diagram
conductor principal musician
rhythm[]
rhythm{Forevery event in the Rhythmpaly the event}
next cycle
synchronization
log
rhythm[]
rhythmlog
Index:0
Index:12, 3 ...
next cycle
~microsecond
~microsecond
duration
25
Component specification
From the Use Case Model and mind mapping depicted above we can deduce the basic
required components to do the overall functionality of the orchestra application; hence, the
following components are indentified:
1. The conductor component
2. The principal component
3. The musician component
A summarized number of blocks and components required and functionality.
Sub-application/
Component (FASA)
Functional Block
/Block (FASA)
Functionality
1xconductorSA 1xconductorFB - Parse Midi (orchestra)
- Divide the midi into group of tracks
- Identify set of tracks based on manual conf.
- Distribute (rhythm, index, duration message )
to sub-conductors/principals
4xprincipalSA 4xprincipalFB - Divide the rhythm among the musicians based
on preconfigured information.
12xmusicianSA 12xmusicianFB - play the rhythm for the allocated duration
Table1: Orchestra application blocks and components and functionality
3.4 Distributed orchestra application overview
The following high level design of orchestra application a single path which is conductor-
principal-musician. Likewise, the same procedure will be repeated for multiple principals and
musicians. This single application which is playing an orchestra will be will be modeled and
implemented in the next sub section.
Orc
he
stra
ap
pli
cati
on
Parse midi file
Extract rhythm groups
-tracks and events
Dispatch rhythm groups to principals
Port ?Extract rhythm
Events per track
Dispatch rhythm to musicians
Play rhythmNext cycle
Port?
Orchestra organization XML
midifile
Orchestra organization XML
Table 2: Orchestra application high level design
26
3.5 Component modelling
Finally we are going to model the components based on requirements and high level
design described above using 4DIAC-IDE. First we will describe a step-by-step procedure for
creating the component types using 4DIAC-IDE type management perspective. Respective
snapshots are to be found at Appendix A.
Step #1: Create a new project by the name OrchestraProject and fill appropriate fields.
File->New->New System and fill the project name ‘OrchestraProject’
Next, import the default Pallets for using default libraries from 4DIAC.
Fill information related to the project e.g., type, comment, description …
Go to Windows and show ‘System Manager view’, if not opened in the ‘System
Manager perspective’; the created project will be available there.
Step #2: create the functional blocks
Shift perspective to ‘Type Management->Type navigator view’.
Select and right-mouse click on OrchestrProject and create a new folder,
orchestra; it can be any name for that matter.
Right-mouse click on the new folder and go to ‘New Type’
Give a name to the block e.g., conductorFB of ‘Type’ basic block.
Create input and output variables as indicated in Appendix
Repeat this for principal and musician basic blocks (aka functional block in IEC
61488 standard naming)
Step #3: Create orchestra application and use the blocks created in step#2
Go to ‘System Manager’ and right-mouse click and select ‘New application’
Give it a name ‘orchestra’
Drag and drop the blocks created
Connect the event and data channels
(Note: Since device and resource allocation is not supported by 4diac-fasa
transformation plug-in, manual configuration is done. This is only to indicate the work flow
of 4DIAC-IDE). The following diagram is a result of a subsequent steps discussed above.
Following orchestra component modelling on 4DIAC-IDE, let’s have a look on the code
or template generated by 4diac-fasa transformation plug-in.
Figure 19: conductor functional block Figure 17: principal functional block
Figure 18: musician
functional block
27
3.6 Template generated by 4diac-fasa plug-in
The transformation plug-in generates a template for the application developer to start
with writing algorithms. The template is a collection of directories within are found
configuration files and C++ source and header files. These are discussed in detail below:
Application directory
This directory holds XML files for each sub application or components modeled in the
4DIAC IDE. These XML files describe a component such as name and related component
implementation path. According to modeling, there are 15 XML files that describe a
component (1xconductorSA, 2xprincipalSA and 12xmusicianSA). Other XML files includes:
and a single XML for a single host the kernel will be executing.
- Orchestra.xml –a communication and schedule configuration on a single host.
- Orchestra_net_master.XML – a communication and schedule configuration for
the conductor when application is distributed across multiple virtual nodes.
- Orchestra_net_swartxx.xml – a communication and schedule configuration for
principal and musician when application is distributed across virtual nodes.
- Swartxx_net_receive/send.xml – a configuration setup for receiving and
sending proxies; xx refers to host name.
Library directory
This directory holds a template for actual C++ source and header files on a separate
folder for blocks and components.
- Methods – algorithm which describes the behavior of a block
- Variables and State – input/output of blocks and state of a block.
28
Module directory
This directory holds a single configuration file which describes location of each
component and block in the application. A compiler refers this configuration for reference of
blocks and components.
#include "conductorFB.h" #include <iostream> #include <cstdlib> namespace FASA { namespace Libraries { namespace conductorFB { void conductorFB::operator () () { //algorithm } void conductorFB::construct () { FASA_REGISTER_OUT_PORT
(OM); FASA_REGISTER_OUT_PORT
(MDO); } } // namespace masterapp } // namespace Libraries } // namespace FASA
#include "conductorSA.h" #include <iostream> using namespace std; #include "../blocks/conductorFB.cc" namespace FASA { namespace Libraries { namespace conductorSA { conductorSA_Component::conductorFB
_Component (const ID &id, const std::vector<std::string> ¶meters,
Model::Component::State *state) : Component (id, parameters, state) { blocks.set ("conductorFB", new
conductorFB("conductorFB", this)); } void conductorSA _Component::init () { } } // namespace conductorSA } // namespace Libraries } // namespace FASA
Figure 20: conductorFB and conductorSA C++ templates
3.7 Standard Midi file format and partitioning
Standard MIDI file format
MIDI (Musical Instrument Digital Interface) is a technical standard published by
Association of Musical Electronics Industry (AMEI) which defines a protocol, digital
interfacing and communication of musical instruments, computer system and other devices.
MIDI uses event message to communicate among digital midi devices which basically
specifies music notation, pitch, velocity and others. One way of carrying these messages is
using MIDI file as compared to midi sequencers which also is used to edit and play music.
The format of MIDI file used in the scope of works is produced by International Midi
Association, EMI [19]. The file block is made up of chunks and events. Only relevant parts are
discussed below:
Chunk – is a sequence of 8-bit byte with the following format.
<type:4byte><length:4byte>:<data:lengthofdata>
There are two types of chunks: Header Chunk and Track chunk identified by the
keyword type as ASCII character “MThd” and “MTrk”, respectively. Header
Chunk specifies basic information about the data in the midi file; such as number
of tracks and other information. Whereas, Track Chunks the actual chunk which
holds the musical song in a form of events.
Event – are part of a data section in a Track chunk. There is variable number of
events in a track; for instance, middle C key in a piano can be represented as a
29
single event followed by D key. The time difference between the two notes is
called delta-time. The unit of measurement can be ticks or milliseconds.
Event: <delta-time><event>
There are three event types: meta-event, data-event and sysex events. Actual
songs are specified in data-events [19].
MIDI file block: CHUNK
Type Length Data
MThd 6 …<track>…
MTrk <length> <delta-time><event>
….
MTrk <length> <delta-time><event>
Table 3: midi file format
Software tools and libraries
Midi file simply holds MIDI notation and controls; therefore, once parsed the midi file. A
series of steps is required to produce the actual sound from the speaker. Following are the
tools used:
Fluidsynth – is an open source software synthesizer27 which converts midi data
into audio signal using SoundFont technology28. The orchestra application uses
fluidsyth development API to produce sounds based on a midi note data. Please
PulseAudio 29– is the default sound server in Ubuntu which accepts sound
requests from multiple applications for concurrent access and other advanced
audio processing services. For example, the orchestra application and
Rhythmbox application (player in Ubuntu) use the sound card concurrently
through PulseAudio.
ALSA30 – PulseAudio does not contain any drivers for underlying sound cards;
instead, it uses Advanced Linux Sound Architecture, ALSA to communicate with
sound cards. ALSA is the default audio infrastructure in Ubuntu OS and has
supports for many sound card vendors31 as compared to its predecessor OSS32.
27 Software which imitates and produces sounds of instruments using programming algorithms. 28 Maps recorded and synthesized audio; especially for music composition. 29 PulseAudio: http://www.freedesktop.org/wiki/Software/PulseAudio/ 30 ALSA: http://www.alsa-project.org/main/index.php/Main_Page. Aug 14, 2013 31 http://www.alsa-project.org/main/index.php/Matrix:Main#ALSA_SoundCard_Matrix 32 OSS: http://www.opensound.com/oss.html. Aug14, 2013
Sound Card (Intel HDA)
ALSA
PulseAudio
Others App
Orchestra application
fluid
synt
h
Figure 21: Orchestra application audio platform on Ubuntu OS
30
Midi file partitioning and rhythm message
The midi file is partitioned into a set of events per track (also called rhythm in the scope
of work). The even types we are interested in are ON_event and OFF_event. ON_event
indicates start of playing a key note; and OFF_event indicates end of playing a key note. The
orchestra application uses fluid_synth_noteon() and fluid_synth_noteoff() APIs from
playRythm() routine to enforce ON_even and OFF_event on midi driver through synthesizer.
/* Rhythm information */ struct Rhythm { MFEvent *mfevent; int track; int start_index; int end_index; int absolute_tick; int absolute_tick_played; int cycle; int ticksPerQuarterNote; };
Description -Midi events -Track number -Start of event index in midi -End of event index in mid -Start of tick number in the midi -Absolute tick plus duration in tick -The number of cyclic execution -Number of ticks needed to sound a quarter note, used for conversion from tick to seconds
Table 4: Rhythm message format
A rhythm message is a message transmitted by the conductorFB block to the musicianFB
through principalFB. It is used to direct the musician what to play and for how long to play.
This message is sent periodically according to cyclic execution of FASA blocks. Since FASA
modeling is periodic, any variable number of events is treated by the musician within that
period.
Mid
i eve
nts
pa
rtit
ion
ing
duration
Start_abs_time = 0
0 1
ON_event
OFF_event
Track#1
Track#n
rhythm
Figure 22: midi file partitioned and ON/OFF events
31
Co
nd
uct
or
flo
w d
iag
ram
Extract rhythm group
Dispatch rhythm group to principals
End of a cycle
Orchestra organization XML
midifile
State-event index
-abs time
pri
nci
pa
l flo
w d
iag
ram
Extract rhythm
Dispatch rhythm group to musicians
End of a cycle
Orchestra organization XML
Receive rhythm group from conductor
3.8 Algorithm Implementation for functional blocks
Conductor Functional Blocks implementation
The conductor functional block, conductorFB implements the conductor functionality in
the orchestra. It reads midi file and orchestra organization XML and cyclically dispatches
rhythm group (rhythm message for multiple tracks) to principal functional blocks. It
supports four principal functional blocks, from CH0O to CH3O.
The conductorFB sends
rhythm group according to
predefined prinicipal-port
mappings from orchestra
organization XML file.
Since FASA blocks are
stateless, the conductorFB has a
separate state structure,
conductorFB_Block_State() to hold
rhythm information (index,
absolute tick) of a previous rhythm
message.
Principal Functional Block implementation
The principal functional block, principalFB implements the functionality of sub
conductor in the orchestra. It receives a group of rhythms from the conductor and dispatches
the rhythm based on preconfigured track information from orchestra organization XML.
Table 5: conductor flow diagram
Figure 23: principal functional block implementation
32
Musician Functional Block implementation
The musician Functional block, musicianFB implements the functionality of musician in
the orchestra. It receives a rhythm from principal functional block and plays by referring the
event index and duration from the rhythm message.
Whenever the rhythm player is
encountered with ON or OFF events, there is
always a delta time proceeded to it. And this
delta time is used to delay or hold the note for
that time, event.time. If the musicianFB did
not encounter events for one or more cycles,
the delta time is accumulated for at least the
‘duration x the number of cycles not played’.
And by the time an event arrives the
musicianFB, it tries to hold the note for the
prolonged amount of time which is not
correct. Instead, it is expected to delay since
the the latest musicianFB cycle start. For this
reason, we need a state to save the number of
cycles the block did not encounter events; and
then refresh it once encountered to zero.
pri
nci
pa
l fl
ow
dia
gra
m
Play rhythm
End of a cycle
Receive rhythm from principal
Statecycle_not_pla
yed
Port connected
?
yes
startstart
1 2 cycle
Delay provided by the event: not applicable (delta)
Correct_delay
Correct_delay = delta – duration x cycle
ON event OFF event
duration?
Figure 24: musician functional block implementation
33
Chapter 4
Deployment of Orchestra Application
In this chapter we will be discussing setting up of network of virtualized multicore
environment and deployment of the orchestra application. Specifically the following points
will be covered:
Setting up - Network of Virtualized Multicore environment
Define Virtual Computational Resource
Allocate Components to VCR
Launch application from central terminal or master node
4.1 Setting up - Network of Virtualized Multicore environment
In this section we will describe software packages used to establish a network of
virtualized multicore environment. These software packages, which are motivated in problem
analysis section 1.5, include Xen, open vSwitch, Linux distribution from yocto project and
Libvirt/virt-manager. We will also explain installation and configuration of the software
packages on the selected hardware platform. For a complete installation procedure, refer to
Xen_ovs_yocto_installation.pdf [20].
Host installation: Ubuntu-12.04 LTS
Xen hypervisor requires a VM, Dom0 which manages the rest of VMs, DomUs. Usually
the Dom0 is a variant of Linux, but can also be used from NetBSD or OpenSolaris. The most
commonly used Linux variants that have Dom0 support are found from fedora and Ubuntu
distribution (Please refer33, for a list of kernels which supports Dom0). If a distribution
without Dom0 kernel support is installed, a separate Dom0 installation is expected. Unless,
it is necessary to build a separate Dom0, it is always recommended to use one of the
distributions which support Dom0 kernel from the list indicated above. In this thesis work,
Ubuntu-12.04 LTS is used. It has a long term support from Ubuntu, and also has active
community support. In fact, Fedora distribution is also a good alternative.
33 Dom0 kernels for Xen: http://wiki.xen.org/wiki/Dom0_Kernels_for_Xen
HP Compaque Elite 8300 Ultra-slim Desktop summary
Processor : 4x Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
Memory : 946MB (513MB used)
Operating System : Ubuntu 12.04.2 LTS
Audio Adapter : HDA-Intel - HDA Intel PCH
Table 6: Software technologies
34
The Xen hypervisor boots right after a bootloader34; therefore, it is important to
understand how the boot process is performed. The two main options35 used to boot are:
Basic Input/Output (BIOS) and Unified Extensible Firmware Interface (UEFI). Though
UEFI is the most advanced firmware interface and a promised successor to BIOS, we found
BIOS to be much easier and error prone to boot Xen hypervisor. Another important point
during installation of the host is partitioning the hard disk. The host OS occupies a small
portion of space; and the rest can be used for the VMs. Since, it is hard to determine how
much size is required for VM ahead, a Logical Volume Partition (LVM) is scheme used. This
enables dynamic resizing of partitions in the hard disk after installation which is by far more
advantageous than using static partitioning (without LVM). To achieve this, it is
recommended to make the hard disk a logical disk during installation of the host OS.
Xen hypervisor installation
Xen hypervisor can be obtained either from the Xen project36 or from Ubuntu
repository37. Xen hypervisor from Xen project has to be built from source; and the latest
version is always available. On the other hand, Ubuntu repository provides a software
package which is already build and tested for a specific hardware platform; and usually is
more reliable but not usually latest. During the thesis writing, the latest Xen hypervisor
releases, Xen-4.3 and Xen-4.1, are available from Xen project and Ubuntu repository,
respectively. Due to latest features in Xen-4.3 e.g., open vSwitch hot-plug and latest and long
term support tools stack (XL toolstack), Xen hypervisor is configured, build and installed
from source. The next step would be to install Open vSwitch and configure networking.
Open vSwitch installation
It’s possible to configure networking of virtual machines using a default Linux Bridge38;
however, it lucks advanced features of switches such as flow analysis and QoS. Instead, Open
vSwitch is used, which is an advanced virtual switch specifically designed to networking of
virtual machines without compromising switching performance, especially as compared to
Linux Bridge. As mentioned above, the Xen hypervisor already has a hot-plug support for
open vSwitch. This implies ease of configuration and integration of open vSwitch product to
Xen. But open vSwitch has to be installed like any other Linux software package either from
source or Ubuntu package repository. In this installation, we installed open vSwitch from
Ubuntu repository, which is much easier and sufficient to our needs (most importantly
bridging VM and configuring QoS).
VM installation: yocto VM, Ubuntu-12.04 LTS headless
A Dom0 is used for installation of DomUs. Dom0 hosts different tool stacks for
managing DomUs, e.g, XL, virsh and virt-manager. It is possible to create and manage all
types of Xen VMs (paravirtualized,HVM and PVHVM) using the above mentioned tools. In
our scope of work we managed to create and manage HVM for a reason explained on the
analysis of problem 1.5. The type of VMs (DomUs) used in this installation are from yocto
project, which is an opensource project dedicated for building a Linux distribution mostly for
embedded systems; and Ubuntu-12.04 LTS headless distribution (text mode only). Both
types of DomUs are built with Real-Time (RT) patch. This enables the DomUs machine to be
appropriate for soft real-time applications.
34 A piece of computer program which loads an OS or hypervisor and give services to it, e.g.,
memory-space mapping http://wiki.osdev.org/Bootloader 35 BIOS vs. UEFI: http://www.logic.nl/products/technology/bios-and-efi.aspx 36 Xen project: http://www.xenproject.org/downloads.html 37 Ubuntu package repository: http://packages.ubuntu.com/ 38 Linux bridge: https://help.ubuntu.com/community/NetworkConnectionBridge
35
To build an RT based Linux distribution from yocto project, a certain procedure has to
be followed based on yocto project development process. The development process includes
from simple to advanced approaches; e.g., using a predefined configuration template to
generate a Linux distribution, modifying kernel configuration, modifying kernel source, are
some the activities done on yocto development platform. To start with, it’s recommended to
use a graphical tool, Hob tool, which is used to define requirements and generate a Linux
distribution. This is the method used to create RT Linux distribution which lightweigh
(~500MB) and bootable image.
To obtain a headless Ubuntu-12.04 LTS (text based), it is only a matter of configuration
change on the bootloader through /etc/default/grub file (CMD_LINE_LINUX = “text”) –
which basically tells the kernel to load in text mode, hence, no graphical desktop, operated by
command line. Both approaches are implemented in the installation.
Master_node
Cisco switchswart11
swart11
swart11
OpenvSwitch
swart12
swart13
Core-1
Core-2
Core-3
Swart1Core-1 Swart2
swart21
swart11
swart11
OpenvSwitch
swart22
swart23
Core-1
Core-3
Core-2
Core-1
Figure 25: network topology of host and virtual machines using openvswitch
The above network topology shows how the VM as connected to each other using open
vSwitch and Cisco switch. The host machine swart1 has three VMs swart11, swart12 and
swart13; similarly, the host machine swart2 has three VMs swart21, swart22 and swart23.
During installation and configuration of the software packages, we encountered several
challenges e.g., software incompatibility, BUGS, system crash and so forth. As compared to
modeling and implementation of Orchestra application, it took more time than expected due
to the challenges mentioned above. From our experience, we find it important to carefully
follow installation procedure, document all activities during installation (successful
installation or troubleshooting mechanism and challenges) and ccommunicate with
respected software technology development teams, such as subscribing to a mailing list for
patches, bug reports and advices on using the products. And these type of support can be
found either from Ubuntu distribution or the developers e.g., xenProject for Xen hypervisor.
36
4.2 Define Virtual computation resource
Virtual Computation Resource, VCR with QoS for CPU, memory and network
bandwidth, are computing resource definitions generated by partitioning an underlying and
actual hardware resources for allocation, verification and analysis of application parts. VCRs
allow portability of real-time components or application parts to multiple hardware
architecture while preserving timeliness and predictability of an application; e.g., application
porting from 2 core to 4 core machines. Without prior knowledge of hardware resource, VCR
can be prepared separately.
VCR with QoS for CPU, memory and network BW
Following installation of Xen hypervisor, VM and open vSwitch, we will develop a VCR
with QoS for CPU, memory and network BW.
1. QoS for CPU - Available scheduler types in Xen hypervisor by the time of report
writing: credit scheduler (default), Simple Earlier Deadline First, SEDF and
Borrowed Virtual Time, BVT [21]. Credit scheduler and BVT are based on fair
sharing of CPU in mind; and SEDF as a real-time scheduler has some drawbacks
that impact predictability of VM execution. It fairly distributes slack time on
virtual machines after all other VM finish execution. And this disrupts
predictability of virtual machine execution. Therefore, to increase predictability
of orchestra application, as a quick solution39, Xen scheduler is exempted from
load scheduling VM. Instead, each VM are mapped to a single core using the
following XL toolstack commands:
#create pool to hold the pcpu xl cpupool-create name=\"Pool-1\" sched=\"credit\" xl cpupool-create name=\"Pool-2\" sched=\"credit\" xl cpupool-create name=\"Pool-3\" sched=\"credit\" #remove pcpu from Pool-0 xl cpupool-cpu-remove Pool-0 1 xl cpupool-cpu-remove Pool-0 2 xl cpupool-cpu-remove Pool-0 3 #add the pcpu to the created pool xl cpupool-cpu-add Pool-1 1 xl cpupool-cpu-add Pool-2 2 xl cpupool-cpu-add Pool-3 3
Table 7: Create CPU pools for each core
The above XL commands create four Pools for holding a single core. And
each pool will be mapped to a VM by adding the following line into a VM
configuration file.
Pool =Pool-x” where x {1,2,3}
cpus = “x” Sets affinity of the VM to x {1,2,3}
vcpus = 1 Limits the number of VCPUS the VM uses to 1.
GRUB_CMD_LINE_XEN =
“domain0_min_vcpus=1
dom0_vcpus_pin”
To set affinity of Dom0 to core-0
39 Other solutions include development of real-time scheduler which is out of the thesis work
37
2. QoS for Memory –Memory is allocated to each VM including Dom0 to avoid
performance degrade due to DomUs memory starvation caused by Dom0. By
default, Dom0 is allowed to use free memory (aka known as ballooning40 );
therefore, it is recommended to limit each VM memory usage to a certain level of
memory using the following XL commands:
GRUB_CMD_LINE_XEN=
“dom0_mem=1G,max:1G”
Configure at Xen command line,
limits Dom0 to 1G
Memory = ”512” Limits DomUs to 512M
3. Qos for Network BW – the rate at which the VM communicate is limited using
open vSwitch. The following example configures a Virtual Interface, VIF of some
VM to 10Mbps:
#ovs-vsctl set Interface tap0 ingress_policing_rate=10000
#ovs-vsctl set Interface tap0 ingress_policing_burst=1000
*tap0 – name of VIF connected to VM
The following table shows allocation of CPU, memory (RAM) and network BW to each
component.
Component Machine
(hostname)
pCPU vCPU RAM Network BW
(RxTx)
ConductorSA Master node 1xcore 4 1G 10Mbps
princiapalSA_00 Swart-1 Dom0 1xcore 4 1G 10Mbps
princiapalSA_10 Swart-2 Dom0 1xcore 1 1G 10Mbps
musicianSA_10 Swart11 DomU 1xcore 1 512M 10Mbps
musicianSA_20 Swart11 DomU 1xcore 1 512M 10Mbps
musicianSA_30 Swart11 DomU 1xcore 1 512M 10Mbps
musicianSA_40 Swart12 DomU 1xcore 1 512M 10Mbps
musicianSA_50 Swart13 DomU 1xcore 1 512M 10Mbps
musicianSA_60 Swart21 DomU 1xcore 1 512M 10Mbps
musicianSA_70 Swart22 DomU 1xcore 1 512M 10Mbps
musicianSA_80 Swart23DomU 1xcore 1 512M 10Mbps
musicianSA_90 Swart21 DomU 1xcore 1 512M 10Mbps
musicianSA_11 Swart22 DomU 1xcore 1 512M 10Mbps
musicianSA_12 Swart23DomU 1xcore 1 512M 10Mbps
Table 8: Virtual Computation Resource for components
40 Ballooning is a technique used to dynamically increase memory for Dom0 when needed.
38
Application Schedule and Execution
The orchestra components are executed periodically with the following timing
parameters.
Component Period(ms) Offset(ms)
conductorSA 5 10 50 100 0
principalSA_xx 5 10 50 100 1
musicianSA_xx 5 10 50 100 1
Figure 26: Component execution model
A 1ms offset is sufficient for network latency between the components, hence a total of
2ms between the conductor and musician. This is found from traffic analaysis of openvirtual
switch with an average latency ~0.032ms.
track1
track2
Op
Om
track3
track4
track5
track6
track7
track8
0 2ms 2+d T
track9
track1
track2
track3
track4
track5
track6
track7
track8
T+2ms T+2ms+d 2T
track9
conductorSA
principalSA_00
principalSA_10
musicianSA_10
musicianSA_20
musicianSA_30
musicianSA_40
musicianSA_50
musicianSA_60
musicianSA_70
musicianSA_80
musicianSA_00
Figure 27: orchestra application components trace
39
4.3 Orchestra Application Management
Following allocation of components to virtual computation resource as describe in the
previous section, the application is ready for launching. A script is executed from the master
node which initiates the FASA kernel instances on each machine to run at an absolute time.
All machines in the system are synchronized using Network Time Protocol, NTP protocol.
Master node is the NTP server which intern is synchronized with ntp.ubuntu.com. All guest
machines including machines where the principal components reside are synchronized to the
master node instead of ntp.ubuntu.com to increase precision (because of the geographical
vicinity). The following management tools are used to manage components, virtual machines
and the orchestra application.
FASA component manager
Each kernel instance has a component manager for switching configuration, adding and
removing components, add or remove channels, get list of components. This is a standalone
and also a distribute management framework of FASA.
Libvirt and Virt-manager
This is GUI based on libvirt41 for managing virtual machines. It has a wizard for creating
virtual machines and configuring VM easily. It supports VNC42 and SPICE43 client viewers for
displaying virtual machine console.
Orchestra distribute management script
This is a script developed to manage specifically the orchestra application and the virtual
multicore environment in distributed mode.
41 It is hypervisor independent library for managing virtual machines and hypervisor. 42 Virtual Network Computing is a graphical desktop sharing using remote buffering.
http://en.wikipedia.org/wiki/Virtual_Network_Computing 43 Simple Protocol for Independent Computing Environment is a remote display system
http://en.wikipedia.org/wiki/SPICE_(protocol)
Orchestra distributed management script
Machines-applies to virtual machines
and hosts-create, list, reboot VM
-configure network-issue any command-distributed sshkey
Application-compile
-update machines-run application
Figure 28: orchestra management script features
40
Chapter 5
CONCLUSION
In the previous subsequent chapters of the thesis report, we modelled and implemented
an orchestra application mainly using 4DIAC-IDE for demonstrating and evaluating FASA
component framework on a virtualized multicore environment. And we defined the concept
of a virtual computational resource (VCR) for components. Based on the VCR defined, the
application was deployed on virtualized multicore PCs. In the following sub sections, we will
discuss the result and future work.
5.1 Result
The result will be discussed in accordance to sub problems outlined in the problem
formulation, chapter 1.3.
Problem#1:
Model and implement an orchestra application on a network of virtualized multicore
In chapter 3, we modeled and implemented Orchestra application mainly using 4DIAC-
IDE. First orchestra music and organization structure was studied briefly; followed by,
requirement analysis and modeling using various tools but mainly using 4DIAC-IDE based
on IEC61499 functional blocks. The model is deployed on FASA runtime environment or
kernel where timeliness and predictability of orchestra application was performed.
The functionality of the components was implemented first by studying standard MIDI
format and partitioning it into manageable rhythm messages. Then the algorithm was
defined for each component types (conductor, principal and musician) and tested to work as
expected. Finally, we connected all components as a bottom-up fashion to construct the
application by defining resources on all devices such as communication channels, CPU
schedule and execution sequence.
Problem #2:
Study and implement the concept of Virtual Computational Resource, VCR with QoS for
CPU, memory and network bandwidth.
First the concept of VCR was studied in chapter 4.1 and then using Xen hypervisor we
were able to virtualize physical resources such as memory, CPU and network bandwidth.
Virtual Machines of type Hardware-assisted Virtual Machines (HVM) were created.
Following that we defined QoS for CPU and memory using XL toolstack and virt-manager for
each virtual machine (aka DomUs) which enabled to achieve special and temporal isolation of
virtual machines. We used open vSwitch to define QoS measured in Mbps for network
communication between virtual machines for transmitting and receiving rates. And network
throughput, latency and jitter of virtual machines was measured using iperf tool on a virtual
and physical network connected using open vSwitch and Cisco switch, respectively. And the
result shows no datagram lost in 32768 bytes transferred within 10 seconds from a host to
virtual machine. And latency was reported by the receiving end was around 0.005ms. For
more, refer to Appendix F and G.
41
Problem #3:
Study to what degree the architecture will support software independence by
evaluating how different allocations of application parts impact real-time behavior.
The application is tested in the same hardware architecture by enabling and disabling
physical cores; and the orchestra performed as before. However, it is not ported on different
hardware platform and tested due to time limitation. But we believe it will perform the same
as long as the required Virtual Computational Resource is available in the new hardware
platform and using Xen hypervisor as virtualization enabler.
Problem #4:
Dynamic turn ON/OFF of instruments and prepare test configurations.
A set of configuration is created with different QoS to evaluate the response of the real-
time component-based development platform. For more single host component allocation,
Appendix A and network level component allocation, Appendix C.
5.2 Future work
Future work of the thesis can be viewed from two perspectives. First we can extend the
application for evaluation of the FASA framework. Second we can improve the definition of a
virtual computational resource. These two are studied in more detail bellow:
Extending orchestra application and development
1. Currently each VM is allocated a single component. In the future we could play
multiple instruments (designed as components) in a single virtual machine. This
would mean that a virtual machine has to interleave the musician blocks to play
the music, similarly on other VMs without any drift from the timeline of the
orchestra. This gives more loads to the real-time component execution
framework on hierarchical scheduler nature (scheduler by the hypervisor and
VM RTOS).
2. Add a midi instrument support e.g., keyboard directly connected to midi
interface; and GUI interface for selecting orchestra, dynamic turn ON/OFF of
tracks.
3. Integrate FASA runtime environment similar to FORTE in the 4DIAC-IDE. This
could ease the development of application for FASA framework.
4. Evaluating other real-time component-based frameworks using the orchestra
application.
Defining virtual computational resource
Currently Xen lucks an offline real-time scheduler. In the future, Xen can be
extended with an offline and static scheduler that can be used for scheduling
control applications.
In the future the concept of a virtual computational resource could be defined as
a separate layer regardless of which real-time component execution framework is
used.
42
Chapter 6
REFERENCES
[1] ABB, "System 800xA System Introduction technical document for customers,"
2012.
[2] M. Oriol, R. Steiger, S. Stoeter, E. Vardar, M. Wahler, H. Koziolek and A. Kumar,
"FASA: a scalable software framework for distributed control systems," in
Proceedings of the 3rd international ACM SIGSOFT symposium on Architecting
Critical Systems, Bertinoro, Italy, 2012.
[3] K. Ihor, L. Yan, G. Ian and H. Gernot, "G.: CAmkES: a component model for secure
microkernel-based embedded systems," 2007.
[4] T. Genßler, A. Christoph, M. Winter, O. Nierstrasz, S. Ducasse, R. Wuyts, G.
Arévalo, B. Schönhage, P. Müller and C. Stich, "Components for embedded
software: the PECOS approach," Proceedings of the 2002 international conference
on Compilers, architecture, and synthesis for embedded systems, Vols. 1-58113-
575-0, no. CASES '02, pp. 19--26, 2002.
[5] J. Chouinard, J. Lavallée, J.-F. Laliberté, N. Landreaud, K. Thramboulidis, P.
Bettez-Poirier, F. Desy, F. Darveau, N. Gendron and C.-D. Trang, "An IEC 61499
configuration with 70 controllers; challenges, benefits and a discussion on
technical decisions," in 14th IEEE Conference ETFA’07, Patras, 2007.
[6] X. Sisu, W. Justin, L. Chenyang and G. Christopher, "RT-Xen: Real-time
virtualization based on hierarchical scheduling," in Embedded Software
(EMSOFT), 2011 Proceedings of the International Conference on. IEEE, 2011.,
2011.
[7] L. Min, A. S. Krishnakumar, P. Krishnan, S. Navjot and Y. Shalini, "Supporting soft
real-time tasks in the xen hypervisor," in Proceedings of the 6th ACM
SIGPLAN/SIGOPS international conference on Virtual execution environments,
Pittsburgh, Pennsylvania, USA, 2010.
[8] A. Moller, M. Akerholm, J. Fredriksson and M. Nolin, "Evaluation of component
technologies with respect to industrial requirements," Euromicro Conference,
2004. Proceedings. 30th, pp. pp.56,63, 31 , Aug.-3 Sept. 2004.
[9] E. de Almeida, A. Alvaro, D. Lucredio, A. do Prado and L. Trevelin, "Distributed
component-based software development: an incremental approach," in Computer
Software and Applications Conference, 2004. COMPSAC 2004. Proceedings of
the 28th Annual International, 28-30 Sept. 2004.
[10] I. Crnkovic and M. Larsson, "The Definition and Specification of Components," in
Building Reliable Component-Based Software Systems, Norwood, Artech House,
2003, p. 1.
[11] W. Michael, S. Richter, S. Kumar and M. Oriol, "Non-disruptive large-scale
component updates for real-time controllers," in ." In Data Engineering In Data
43
Engineering Workshops (ICDEW), 2011 IEEE 27th International Conference on,
pp. 174-178, 2011.
[12] S. Richter, M. Wahler and A. Kumar, "A Framework for Component-Based Real-
Time Control Applications," in In 13th Real-Time Linux Workshop, Prague, Czech,
2011.
[13] M. Wahler, S. Richter and M. Oriol, "Dynamic software updates for real-time
systems," in In Proceedings of the 2nd International Workshop on Hot Topics in
Software Upgrades (HotSWUp '09), ACM, New York, NY, USA, , Article 2 , 6
pages, July 31, 2007..
[14] IBM Global Education White Paper, "Virtualization in Education," October 2007.
[15] A. J. Younge, R. Henschel, J. T. Brown, G. v. Laszewski and G. C. F. Judy. Qiu, "A
performance Analysis of Xen and KVM hypervisors for hosting Xen Worlds
Project," in 2011 IEEE 4th International Conference on Cloud Computing, 2011.
[16] S. B. Pfaff, J. Pettit, T. Koponen, K. Amidon and M. C. Scott, "Extending
networking into the virtualization layer," in 8th ACM Workshop on Hot Topics
inNetworks (HotNets-VIII), New YorkCity,NY, October 2009.
[17] J. H. Christensen, T. Strasser, A. Valentini, V. Vyatkin and A. Zoitl, "The IEC 61499
Function Block Standard Overview Of The Second Edition," ISA Automation ,
week 6 2012.
[18] V. Vyatkin, "The IEC 61499 standard and its semantics," in Industrial Electronics
Magazine, IEEE , vol.3, no.4, Dec. 2009.
[19] The Internal MIDI Association, "Standard MIDI file format Spec. 1.1," The Internal
MIDI Association, Los Angeles.
[20] M. Nesredin, "Xen hypervisor installation and Yocto Machines," AT, ABB CRC,
Västerås, 2013.
[21] C. Ludmila, G. Diwaker and V. Amin, "Comparison of the three CPU schedulers in
Xen," SIGMETRICS Perform. Eval. Rev., vol. 35, no. 2, pp. 42--51, sep 2007.
[22] T. Strasser, "Framework for Distributed Industrial Automation and Control
(4DIAC)," in Industrial Informatics, 2008. INDIN 2008. 6th IEEE International
Conference on, Daejeon, 2008.
[23] VMware, "Virtualization in Industrial Automation," 2010.
[24] ABB Company, "800xA solution Handbook," ABB, Sweden, 2013.
[25] Intel Whitepaper, "Applying Multi-core and Virtualization to Industrial and
Safety-Related Applications," 2009.
[26] Lago, N. Posse and F. Kon, "The Quest for Low Latency," in PROCEEDINGS OF
THE INTERNATIONAL COMPUTER MUSIC CONFERENCE (ICMC2004}, 2004.
[27] W. Shengquan, S. Rho, M. Zhibin, R. Bettati and Z. Wei, "Real-time component-
based systems," in Real Time and Embedded Technology and Applications
Symposium, 2005.RTAS 2005. 11th IEEE , vol., no., 7-10 March 2005.
1
Chapter 7 Appendix
A. Orchestra modeling using 4DIAC
a. Step #1: Creating orchestra project
b. Step #2: create a new folder for the new blocks
2
c. Step#3: create new functional blocks and new application (to the left)
d. Step#3.1:drag and drop functional blocks
3
Step#3.2: convert into sub applications (in FASA term: into components)
e. Step#4: transform the application using 4diac-fasa plug-in
(Install the plug-in just like any other eclipse plug-in)
f. Write the algorithm of the functional block in Eclipse platform
g. Compile, debug and run it on FASA component runtime environment
4
B. Orchestra component resource allocation on a single host and scheduling
<?xml version="1.0" encoding="utf-8"?>
<application>
<components>
<component id="conductorSA_00" description="orchestra/conductorSA.xml" />
<component id="principalSA_00" description="orchestra/principalSA_00.xml" />
<component id="musicianSA_10" description="orchestra/musicianSA_10.xml" />
<component id="musicianSA_00" description="orchestra/musicianSA_00.xml" />
<component id="musicianSA_10" description="orchestra/musicianSA_10.xml" />
<component id="musicianSA_20" description="orchestra/musicianSA_20.xml" />
<component id="musicianSA_30" description="orchestra/musicianSA_30.xml" />
</components>
<channels>
<!-- LEVEL 0: conductor -->
<channel id="channel_1" size="100">
<source component="conductorSA" block="conductorFB" port="CH0O" />
<destination component="principalSA_00" block="principalFB_00" port="CH0I" />
</channel>
<!-- LEVEL 1: principal -->
<channel id="channel_11" size="100">
<source component="principalSA_00" block="principalFB_00" port="CH0O" />
<destination component="musicianSA_00" block="musicianFB_00" port="CH0I" />
</channel>
<channel id="channel_12" size="100">
<source component="principalSA_00" block="principalFB_00" port="CH1O" />
<destination component="musicianSA_10" block="musicianFB_10" port="CH0I" />
</channel>
<channel id="channel_13" size="100">
<source component="principalSA_00" block="principalFB_00" port="CH2O" />
<destination component="musicianSA_20" block="musicianFB_20" port="CH0I" />
</channel>
<channel id="channel_14" size="100">
<source component="principalSA_00" block="principalFB_00" port="CH3O" />
<destination component="musicianSA_30" block="musicianFB_30" port="CH0I" />
</channel>
</channels>
5
<schedule>
<sequential>
<execute component="conductorSA" block="conductorFB" />
<execute component="principalSA_0" block="principalFB_00" />
<execute component="musicianSA_0" block="musicianFB_00" />
<!-- <execute component="musicianSA_1" block="musicianFB_10" />-->
</sequential>
</schedule>
</application>
C. Orchestra component resource allocation on a network level and scheduling
<?xml version="1.0" encoding="utf-8"?>
<application>
<components>
<component id="conductorSA" description="orchestra/conductorSA.xml" />
<component id="principalSA_0" description="orchestra/principalSA_0.xml" />
<component id="NetProxy_Send_00" description="orchestra/swart11_net_send.xml" />
<component id="NetProxy_Send_01" description="orchestra/swart12_net_send.xml" />
<component id="NetProxy_Send_02" description="orchestra/swart41_net_send.xml" />
<component id="NetProxy_Send_03" description="orchestra/swart42_net_send.xml" />
</components>
<channels>
<!-- conductor to principal channels-->
<channel id="channel_1" size="100">
<source component="conductorSA" block="conductorFB" port="CH0O" />
<destination component="principalSA_0" block="principalFB_00" port="CH0I" />
</channel>
<!-- principal to musicians channels -->
<channel id="channel_11" size="100">
<source component="principalSA_0" block="principalFB_00" port="CH0O" />
<destination component="NetProxy_Send_00" block="send" port="NetProxy_Send_In" />
</channel>
<channel id="channel_12" size="100">
6
<source component="principalSA_0" block="principalFB_00" port="CH1O" />
<destination component="NetProxy_Send_01" block="send" port="NetProxy_Send_In" />
</channel>
<channel id="channel_13" size="100">
<source component="principalSA_0" block="principalFB_00" port="CH2O" />
<destination component="NetProxy_Send_02" block="send" port="NetProxy_Send_In" />
</channel>
<channel id="channel_14" size="100">
<source component="principalSA_0" block="principalFB_00" port="CH3O" />
<destination component="NetProxy_Send_03" block="send" port="NetProxy_Send_In" />
</channel>
</channels>
<schedule>
<sequential>
<execute component="conductorSA" block="conductorFB" />
<execute component="principalSA_0" block="principalFB_00" />
<execute component="NetProxy_Send_00" block="send" />
<execute component="NetProxy_Send_01" block="send" />
<execute component="NetProxy_Send_02" block="send" />
<execute component="NetProxy_Send_03" block="send" />
</sequential>
</schedule>
</application>
7
D. Virtual machine configuration xen hyprevisor
# =====================================================================
# Example HVM guest configuration
# =====================================================================
# This is a fairly minimal example of what is required for an
# HVM guest. For a more complete guide see xl.cfg(5)
# This configures an HVM rather than PV guest
builder = "hvm"
# Guest name
name = "swart11-hvm"
# 128-bit UUID for the domain as a hexadecimal number.
# Use "uuidgen" to generate one if required.
# The default behavior is to generate a new UUID each time the guest is started.
#uuid = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX"
#########################
# CPU SCHEDULING
##########################
#nuber of vcpus
vcpus = 1
#pin the guest machine to core-1
cpus = "1"
###########################
# MEMORY ALLOCATION (MB)
###########################
memory = 1024
#maxmem = 512
###########################
# EVENT ACTIONS
###########################
on_poweroff = "preserve"
#on_reboot = "restart"
#on_watchdog = "destrory"
#on_crash = "destroy"
###########################
# DEVICES
###########################
8
# Network devices
vif = [ 'mac=00:16:3E:74:3d:01, model=e1000, bridge=xenbr0,type=ioemu' ]
# Disk
#disk = [ '/dev/swart-1/swart-11,,hda', '/home/swart/ubuntu-12.04.2-desktop-amd64.iso,,hdc,cdrom' ]
#pci=[ '00:1b.0' ]
disk = [ '/dev/swart-1/swart-11,,hda' ]
boot = "c"
# Guest VGA console configuration, either SDL or VNC
#sdl = 1
vnc = 1
vnclisten = "0.0.0.0"
vncdisplay = 1
soundhw = "hda"
device_model_version = 'qemu-xen'
device_model_override = '/home/swart/qemu/i386-softmmu/qemu-system-i386'
E. HP Compaque Elite 8300 Ultra-slim Desktop summary
Processor : 4x Intel(R) Core(TM) i5-3470S CPU @ 2.90GHz
Memory : 946MB (513MB used)
Operating System : Ubuntu 12.04.2 LTS
Audio Adapter : HDA-Intel - HDA Intel PCH
1
F. Open vSwitch throughput test result using iperf tool Senario #1: Dom0 throughput (Traffic is send and received within Dom0)
sender
receiver
------------------------------------------------------------------------------------------------------
Interval Transfer Bandwidth
Interval Transfer Bandwidth
0.0-20.0 sec 78978 MBytes 33.1 Gbits/sec
0.0-20.0 sec 78978 MBytes 33120 Mbits/sec
0.0-20.0 sec 77031 MBytes 32.3 Gbits/sec
0.0-20.0 sec 77031 MBytes 32307 Mbits/sec
0.0-20.0 sec 67910 MBytes 30.7 Gbits/sec
0.0-20.0 sec 67910 MBytes 28471 Mbits/sec
0.0-20.0 sec 79585 MBytes 28.5 Gbits/sec
0.0-20.0 sec 79585 MBytes 33364 Mbits/sec
0.0-20.0 sec 78061 MBytes 32.7 Gbits/sec
0.0-20.0 sec 78061 MBytes 32738 Mbits/secsenario #2: DomU throughput
(Traffic is send from swart1 to swart11 virtual machine)
sender
receiver
-------------------------------------------------------------------------------------------------------
Client connecting to 127.0.0.1, TCP port 5001
Server listening on TCP port 5001
TCP window size: 0.16 MByte (default)
TCP window size: 0.08 MByte (default)
-------------------------------------------------------------------------------------------------------
[ ID] Interval Transfer Bandwidth
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-20.0 sec 53338 MBytes 22372 Mbits/sec
[ 4] 0.0-20.0 sec 53338 MBytes 22363 Mbits/sec
[ 3] 0.0-20.0 sec 48334 MBytes 20273 Mbits/sec
[ 5] 0.0-20.0 sec 48334 MBytes 20266 Mbits/sec
[ 3] 0.0-20.0 sec 50476 MBytes 21171 Mbits/sec
[ 4] 0.0-20.0 sec 50476 MBytes 21168 Mbits/sec
[ 3] 0.0-20.0 sec 48342 MBytes 20276 Mbits/sec
[ 5] 0.0-20.0 sec 48342 MBytes 20271 Mbits/sec
[ 3] 0.0-20.0 sec 50179 MBytes 21046 Mbits/sec
[ 4] 0.0-20.0 sec 50179 MBytes 21041 Mbits/sec
senario #3: DomU <-> DomU (traffic send from swart11 to swart12 virtual machines)
sender
receiver
----------------------------------------------------------------------------------------------------------
Client connecting to 10.10.15.95, TCP port 5001
Server listening on TCP port 5001
TCP window size: 0.02 MByte (default)
TCP window size: 0.08 MByte (default)
----------------------------------------------------------------------------------------------------------
[ ID] Interval Transfer Bandwidth
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-20.0 sec 14565 MBytes 6109 Mbits/sec
[ 4] 0.0-20.0 sec 14565 MBytes 6108 Mbits/sec
[ 3] 0.0-20.0 sec 14744 MBytes 6184 Mbits/sec
[ 5] 0.0-20.0 sec 14744 MBytes 6183 Mbits/sec
[ 3] 0.0-20.0 sec 14704 MBytes 6167 Mbits/sec
[ 4] 0.0-20.0 sec 14704 MBytes 6164 Mbits/sec
[ 3] 0.0-20.0 sec 14672 MBytes 6154 Mbits/sec
[ 5] 0.0-20.0 sec 14672 MBytes 6152 Mbits/sec
[ 3] 0.0-20.0 sec 14540 MBytes 6098 Mbits/sec
[ 4] 0.0-20.0 sec 14540 MBytes 6098 Mbits/sec
1
G. Open vSwitch – jitter and latency test
Server listening on UDP port 5001
Receiving 32768 byte datagrams
UDP buffer size: 256 KByte (WARNING: requested 128 KByte)
------------------------------------------------------------
[ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 45088
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.043 ms 0/ 38 (0%)
[ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.054 ms 0/ 38 (0%)
[ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.044 ms 0/ 38 (0%)
[ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.041 ms 0/ 38 (0%)
[ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.036 ms 0/ 38 (0%)
[ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.087 ms 0/ 38 (0%)
[ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.091 ms 0/ 39 (0%)
[ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%)
[ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.098 ms 0/ 38 (0%)
[ 3] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.094 ms 0/ 38 (0%)
[ 3] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.103 ms 0/ 383 (0%)
[ 4] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 37397
[ 4] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.089 ms 0/ 38 (0%)
[ 4] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.103 ms 0/ 38 (0%)
[ 4] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%)
[ 4] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.108 ms 0/ 38 (0%)
[ 4] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%)
[ 4] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.106 ms 0/ 38 (0%)
[ 4] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.106 ms 0/ 39 (0%)
[ 4] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%)
[ 4] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.091 ms 0/ 38 (0%)
[ 4] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.111 ms 0/ 38 (0%)
[ 4] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.117 ms 0/ 383 (0%)
383 (0%)
[ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 40925
[ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.032 ms 0/ 38 (0%)
[ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.059 ms 0/ 38 (0%)
[ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.107 ms 0/ 38 (0%)
[ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.097 ms 0/ 38 (0%)
[ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.104 ms 0/ 38 (0%)
[ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.089 ms 0/ 38 (0%)
[ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.089 ms 0/ 39 (0%)
[ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.094 ms 0/ 38 (0%)
[ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.117 ms 0/ 38 (0%)
[ 3] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.116 ms 0/ 38 (0%)
[ 3] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.116 ms 0/ 383 (0%)
[ 4] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 49472
[ 4] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.080 ms 0/ 38 (0%)
[ 4] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.091 ms 0/ 38 (0%)
[ 4] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.092 ms 0/ 38 (0%)
[ 4] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.100 ms 0/ 38 (0%)
[ 4] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.132 ms 0/ 38 (0%)
[ 4] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.104 ms 0/ 38 (0%)
[ 4] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.128 ms 0/ 39 (0%)
[ 4] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.086 ms 0/ 38 (0%)
[ 4] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.105 ms 0/ 38 (0%)
[ 4] 9.0-10.0 sec 1.19 MBytes 9.96 Mbits/sec 0.202 ms 0/ 38 (0%)
[ 4] 0.0-10.0 sec 12.0 MBytes 10.0 Mbits/sec 0.189 ms 0/ 383 (0%)
[ 3] local 10.10.15.117 port 5001 connected with 10.10.15.130 port 37970
[ 3] 0.0- 1.0 sec 1.19 MBytes 9.96 Mbits/sec 0.099 ms 0/ 38 (0%)
[ 3] 1.0- 2.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38 (0%)
[ 3] 2.0- 3.0 sec 1.19 MBytes 9.96 Mbits/sec 0.107 ms 0/ 38 (0%)
[ 3] 3.0- 4.0 sec 1.19 MBytes 9.96 Mbits/sec 0.099 ms 0/ 38 (0%)
[ 3] 4.0- 5.0 sec 1.19 MBytes 9.96 Mbits/sec 0.121 ms 0/ 38 (0%)
[ 3] 5.0- 6.0 sec 1.19 MBytes 9.96 Mbits/sec 0.130 ms 0/ 38 (0%)
[ 3] 6.0- 7.0 sec 1.22 MBytes 10.2 Mbits/sec 0.090 ms 0/ 39 (0%)
[ 3] 7.0- 8.0 sec 1.19 MBytes 9.96 Mbits/sec 0.112 ms 0/ 38 (0%)
[ 3] 8.0- 9.0 sec 1.19 MBytes 9.96 Mbits/sec 0.101 ms 0/ 38
