Virtual testing and evaluation with Simware and the Internet of Simulations

Reference: Simware Technical Library

Date: July-2017

Version : 1

© SIMWARE SOLUTIONS S.L., 2017. All rights reserved.

Technical Resources [ Developing Military Test & Evaluation

Simulation Networks with Simware ]

SIMWARE Technical Resources

Developing T&E sim network with Simware Date: July 2017

of 19

Property of SIMWARE SOLUTIONS S.L.

[email protected] – www.simware.es

1. OBJECTIVE

This document describes how to leverage Simware platform and its associated

Layered Simulation Architecture to build any kind of Test & Evaluation

distributed simulation network.

2. INTRO TO MILITARY T&E

Net-Centric military operations are evolving to the use of “smart” weapons,

sensors and platforms that work “connected” in a Common battlefield. The

qualification process of the systems involved in Net-Centric warfare forms a

costly part of the development of new concepts and systems. Today, a

substantial portion of this process can be performed through virtual testing

(simulation), avoiding the need for multiple live tests of real hardware, that are

expensive, hazardous and can result in the partial or total loss of the tested

equipment (total in the case of munition).

Traditionally, the test & evaluation facilities and its associated systems are

created as “stovepipe” systems, built with different suites of proprietary sensors,

networks, hardware, and software.

This “stovepipe” and proprietary approach, even in the cases when it can be

applied, as in the case of T&E activities for a standalone system (not connected

to others in the network), implies an increase in the complexity and costs of the

project, because they use to be based on ad-hoc models, tools and

simulations, developed on a case-by-case basis to enable the test and

evaluation of the system.

But stovepipe and ad-hoc solutions are not valid for the case of “smart”

weapons that operate autonomously or semi-autonomously as part of a system

of systems, composed by multiple platforms, sensors and operators connected

by an integrated command and control network. In this case models,

simulation and tools to test, evaluate and qualify the systems must be able to

interoperate between them in a distributed virtual environment, in the same

way that the actual systems do.

Then, to support the war-fighting community, interoperability and reuse of

resources within the T&E communities are needed to validate weapon system

performance in a cost-effective manner. Cost-effective integration of

resources from T&E facilities is critical to test and determine the war worthiness

of today’s advanced weapon systems and concepts.

mailto:[email protected]

http://www.simware.es/



of 19



3. REQUIREMENTS FOR A MODERN T&E NETWORK

To meet the requirements for cost-effectiveness, increased interoperability and

distributed simulation in T&E tasks, lead users as the US DoD has been already

working in the development of new capabilities supported by distributed

simulation architectures. The best-known example is TENA, acronym for Test

and training ENabling Architecture. TENA was developed as an alternative to

HLA to provide an enterprise wide architecture and a common software

infrastructure to reuse and interoperate test & training range assets. Other

example is the new Object Simulation framework (OSF), in development by the

US Missile Defense Agency since 2011 as the core test and simulation framework

for elements of the Ballistic Missile Defense System (BMDS) to support full scale

simulations, ground tests and live fire events.

In both cases, main requirements are:

- Large scale simulations operating in a “plug & play” distributed

simulation environment.

- Capability to interoperate live, virtual and constructive assets without

limitations.

- Real time performance.

- An Open and modular Common architecture and tools

- Protect intellectual property rights of M&S providers.

- Enable the reusability of assets in different programmes and exercises.



http://www.tena-sda.org/



of 19



4. WHAT DO YOU NEED TO BUILD A MODERN T&E SOLUTION?

Virtual T&E is about to enable the seamless connectivity of digital models,

simulations, sensors, C4I systems and networks in an integrated synthetic

scenario. This is a typical Internet of Simulations scenario. The Internet of

Simulations or IoS is the generalization of the concept of LVC simulation or Live-

Virtual and Constructive Simulation, as distributed simulation is now known in

the military domain. IoS extends the Internet of Things to the M&S domain,

enabling the seamless interoperability of digital models and simulations with

people, processes and things.

Figure 1 Conceptual view of the Internet of Simulations or IoS

To implement an IoS solution you need a net-centric simulation platform, open

APIs to connect the simulations, people, processes and things together,

capability to use the models and simulations as services, workflow

management capabilities to manage the digital workflow and strict

adherence to standards to assure connectivity between heterogeneous assets

Figure 2 Main artefacts in an IoS solutions

To know further about the Internet of Simulations you can visit our dedicated

microsite in our web.



http://www.simware.es/internet-of-simulations.html



of 19



Application of IoS to the specific case of Test & Evaluation ranges is very

straightforward. As in any other IoS application, we need to:

1. Implement a common net-centric simulation architecture, providing

common services and tools across the whole T&E domain.

2. Encapsulate the models and simulations as services, with standard

based interfaces.

3. Provide APIs and connectivity tools to integrate third-party assets, as:

simulations, operator interfaces, C4I applications or live systems and

sensors. In this case connectivity with common standards in the military

domain as HLA, DIS, DDS or MSDL/CBML is a requirement.

Figure 3 An IoS compliant T&E Simulation network

The best lever you can use to develop an IoS compliant solution as this T&E sim

network is Simware platform. Only Simware portfolio provides, just out of the

box, the foundation to build a fully open, modular and distributed simulation

solution that can be integrated with all kind of user interfaces, things and

processes.





of 19



5. BRIEF INTRODUCTION TO SIMWARE PLATFORM

Simware is a platform to run large-scale simulated worlds in the network. A

simulated world can be any kind of virtual space, from a digital laboratory to

design new vehicles or a synthetic scenario to train collectively military forces,

to a virtual model of the human body or a full simulation of a smart city and all

its connected things.

Simware platform is based on a microservices architecture, named Layered

Simulation Architecture or LSA. LSA is a nominating standard at SISO. LSA is the

first microservices architecture for simulation, specifically designed to support

the development of real time and Net-Centric simulation products. As any

other microservices architecture, LSA allows to decompose the simulation

product into small and easily manageable components. Microservices are

called Entities in Simware and interoperate with other entities by exchanging

data through a distributed simulation runtime infrastructure, that is working as

the ESB (Enterprise Service Bus) of the simulation product.

Figure 4 Simware’s micro-services architecture for Net-Centric simulation

Simware platform provides a loosely coupled architecture, composed by

multiple layers that can work alone or in collaboration, depending on the

project’s requirements. Simware layers provided everything you need to

develop real time or event-driven simulations that can be connected with web

and legacy applications in any kind of simulation & training solution.

Simware entities manage persistent objects called Simulation Objects that

have a behaviour and a state that can change because of events in the

simulation. Events are modelled as Interactions in Simware architecture.

Simulation objects supports inheritance and are composed by attributes.



http://www.simware.es/simware-architecture.html



of 19



Several instances of an object can exist at the same time in the simulated

world. Interactions normally provoke changes in the overall state of the

simulated world or of any instance of an object. Notifications of any change in

the attributes of an Sim object’s instance or of any interactions are made using

a publish-subscribe mechanism. Owners of the instance’s attributes of the

object or the interaction publish the data to a middleware that will transmit the

information to the subscribers connected to the middleware. Ownership at the

level of attribute in the simulation objects provides great flexibility to the

simulation, enabling sophisticated schemes of integration and interoperability.

For example, the position and velocity of a virtual tank can be simulated by an

entity but the position and orientation of the turret can be simulated by another

entity.

Let’s see the foundation of LSA and Simware with the example of a simple IoS

solution in which a virtual world is used to train the AI algorithms of the control

application that it is managing a flock of autonomous tractors to do remote

mowing. Next picture shows a basic simulation model for the virtual tractor,

composed by a FuelPump, a mowerdeck and an engine with a battery.

Figure 5 Example of a Simulation model in Simware

In this example, the simulation is de-composed in 3 entities or micro-services:

one managing the tractor platform and its mowerdeck, other simulating the

engine and a third one simulating the battery. Besides specific simulation





of 19



events as overheating in the engine or change the oil, other interactions are

modelled in this case to command the creation of a new engine instance

when a new virtual tractor is created or to create a new battery instance that

will provide the electrical power to the engine instance. A live entity is added

to this scenario: the RemoteMowing. This is the real control application, not a

simulated entity, which is the controller of a group of autonomous tractors in

the field. This control application is driving remotely a flock of real tractors and

in this virtual world is doing the same function for the virtual tractors in the

scenario.

Figure 6 Deployment of the Simulation in Simware

The Entities or simulation micro-services in Simware can be deployed in any

machine in the network that has the middleware running in the machine or a

network interface to a remote middleware. In this example, the

EconSeriesTractor and TractorEngine entities are running in one machine and

the Battery entity in a second node. In this case, the Battery entity will create a

new instance of a Topterminal battery simulation object (Size75TopTerminal)

and will publish her attributes to the middleware. The TractorEngine simulation

will use the battery’s data and its own algorithms to simulate the behavior of

the instance of the engine. Attributes of the Engine will be used by the

EconSeriesTractor simulation to simulate the movement of the instance of the

Tractor simulation object. Control of the instance of the tractor will be made

from the RemoteMowing entity that is running in another node in the network.

In this case, the RemoteMowing entity is leveraging the features of inheritance

and ownership to attribute level in Simware to be able to control any simulation

object that it is a children of MseriesTractor. MseriesTractor has the control

attributes and RemoteMowing can then publish the control attributes of any

instance of its children objects.





of 19



Middleware in Simware is in charge of creating the virtual messaging channels

between the publishers and subscribers, managing the flow of data between

them based on the QoS parameters established for each publisher and

subscriber.

A unique feature in Simware is that the entities in former example can be built

with different technologies and tools. In this way, co-simulation (cooperation of

heterogeneous simulations in the same application) can be done easily.

Following with the example, Entities EconSeriesTractor and TractorEngine are

built using the runtime infrastructure layer in Simware platform. This is another

layer in Simware, on top of the middleware that provides the common

simulation services requested to simulate physic-based simulations with real

time determinism. This layer provides the same basic artefacts you can find in

any simulation platform for real time simulators (as training devices or hardware

in the loop simulators) but integrated in a Net and Data-centric architecture as

is LSA. In the case of the Battery simulation, the partners in the supply chain that

provide the actual batteries are also responsible to provide the digital models

and simulations for the batteries, with a standard data-centric interface

compliant with the simulation model of the example. In our example, we have

two partners for the batteries, one has chosen to provide the Battery simulation

with an HLA interface and the other as a web service, with a REST based

interface. In both cases, they don’t need to use Simware software, only to

follow Simware’s meta data-models to interoperate with Simware based

simulations. The real control application, RemoteMowing, is a legacy

component that is connected to the virtual world leveraging its DDS interface.

Figure 7 Interoperability with Simware





of 19



Simware provides extensions as Simware LVC and Simware Web to provide the

connectors to these external entities. All entities, regardless of how they are

done, can interoperate in the same virtual world because they share a

common simulation model. In all cases, intellectual property is protected,

because only the data interface is exposed and the software can be provided

as a service, running on a server in the facilities of each partner.

To know further about Simware platform and its architecture you can visit the

product page in our website or read the technical resources that you can

find here.

6. DESIGNING AN IOS COMPLIANT T&E NETWORK.

To get a better understanding of how to apply the Internet of Simulations

concept and the LSA architecture as the foundation to design a Test &

Evaluation simulation architecture we are going to use TENA requirements as

the reference. We use TENA as a reference because it is one of the best-known

architectures for T&E ranges because of its use by the US military forces and

some of their military partners, as UK, Australia, etc.

The 3 main technical requirements in TENA are: interoperability, reusability and

composability:

- In the case of interoperability, TENA goes beyond other simulation

standards as DIS or HLA, and push for semantic interoperability based on

common architecture, processes, language, exchange mechanisms

and data-models.

- To improve reusability, TENA demands, beside the compliance with a

common architecture, well-defined and well-documented interfaces

and the capability to talk a common language.

- TENA demands the ability to rapidly assemble, initialize, test, and execute

a system or system-of-systems from members of a pool of reusable,

interoperable software elements.

All former requirements can be achieved easily by designing a technical

solution compliant with IoS and the LSA architecture.

A data-centric architecture improves the interoperability between

heterogeneous assets. Simulation models in LSA based on simulation objects

and interactions provides a common language to all the assets, regardless of

the technology used in their implementation. Interoperability is also very much

improved with LSA because its support for multiple protocols and architectures



http://www.simware.es/simware-lvc.html

http://www.simware.es/simware-web.html

http://www.simware.es/products.html

http://www.simware.es/resources.html



of 19



in the same solution. Different layers can be added to the middleware to

provide connectors to assets using other protocols as simulations using HLA or

DIS, C4I systems using CBML or MSDL or real sensors using DDS.

Open interfaces improve reusability, because they provide a well-defined and

well documented interface based on a common language: the LSA simulation

model with its simulation objects and interactions. LSA simulation model is

based on HLA object model templates, assuring then a clear understanding of

the models by the main M&S communities.

Figure 8 IoS compliant T&E Simulation network

Composability is much improved with LSA, not only because of the capability

to connect interoperable software elements but also because it allows to de-

compose a complex simulation in small and independent entities or micro-

services. Only LSA adds to the middleware a run-time infrastructure simulation

layer that allow to de-compose a real-time simulation in a set of services, that

can be implemented by different partners. Common simulation services in the

runtime infrastructure assure the synchronized execution of the different

simulation entities with real time determinism, even when the services are

running in different machines. Other distributed simulation architectures as

TENA or HLA only provides a middleware to exchange information but they are

missing a standard real time infrastructure that can be used to manage real

time simulations in the network. Only LSA allows to compose a complex real

time simulation as the integration of several simulation micro-services running in

the network. This improved composability is the only way to avoid the ad-hoc





of 19



and proprietary solutions that are now so common in the T&E products, as

already discussed in section 2.

A high-level design of a T&E virtual network based on LSA is shown in below

picture. Common data-models for the T&E network will be designed based on

the simulation objects and interactions as defined in the LSA simulation model.

Interoperability will be assured for all the entities that are connected to the

middleware and are using the T&E datamodel. Connectivity can be done with

a direct connection to the middleware using an API, through another layer as

the runtime infrastructure for the case of physic-based simulations or through a

connector, that is translating the specific protocol used by the legacy entity to

the common T&E datamodel.

Figure 9 LSA based Test & Evaluation virtual network

In any T&E network, besides the simulations and live systems and sensors,

analysis and replay applications are requested. To provide this capability, data

collectors can be plugged to the middleware, subscribed to the flows of data

in the middleware during the execution of the exercises. Data will be stored in

a data-base and analysis applications can use this data during or after the

exercise to do data analytics or replay the exercises.

To enable reusability, any T&E architecture must define a way to develop and

maintain repositories of assets, with open interfaces, that can be rapidly

assembled to perform experiments. In a rapid assembly, the common data-





of 19



model would be developed using as a baseline the data interfaces for all the

reusable assets used in the experiment.

One important point to highlight here is that this design is fully uncoupled and

protect the intellectual property rights of all the participants. Publishers and

subscribers only know the connection with the middleware and its specific

interface with it. In fact, to increase the security, any entity needs to know only

the part of the datamodel that is related to its publishers and subscribers.

7. BUILDING A T&E NETWORK WITH SIMWARE

The design of the IoS compliant T&E network in Figure 9 is easily implemented

with Simware platform. Simware portfolio provides all the software, APIs and

tools needed to build this kind of distributed solution.

Figure 10 Simware Portfolio

Simware provides visual tools that allows to generate rapidly the structure of the

experiment directly from the simulation data-model. Visual Tools included in

Simware Core license allow to design the simulation data-model from scratch

or based on existing simulation data-models and design the publish-subscribe

interfaces for all the entities. Once the design is completed, the tools create

directly from the data-model, C++ and XML files describing the publish-

subscribe interface of all the Entities. Simware tools create also directly the

instantiation of the integration infrastructure based on the designed simulation

data-model, including the publish-subscribe interfaces to the middleware and

a control application (ACS) that allow to control the state machine of the

simulation and to manage the instances in the simulation exercise.



http://www.simware.es/simware-core.html



of 19



Simware Web extension provides a Web server generator to create

automatically the web interface of the common datamodel. This web server

provides a translation to LTI1 messages of the data-models in Simware.

Simware Record&Play extension provides a generator for the Data Collectors

directly from the data-model. Data will be stored in a relational database using

a SQL interface.

XML interfaces created by the tools in Simware Core define in HLA format the

publish-subscribe interface of each subsystem/component. This XML file, or

Interface Definition File, IDF, in Simware terms, is used by SimDeveloper tool, an

extension to the Core in our portfolio, to do model-driven development of the

subsystems/components in the simulation. SimDeveloper is integrated into

Simulink, allowing to develop, integrate and test the simulations models using

Mathworks products: Simulink, Matlab and StateFlow. Automatic generation of

C++ code directly from the model is provided as a feature in Simdeveloper.

SimDeveloper is also the perfect choice to create and maintain repositories of

simulation assets.

Simware LVC extension provides PowerLink tool to generate automatically

gateways to DIS and HLA simulators and a Gateway SDK to speed up the

development of interfaces to any other standard or proprietary protocol, as

DDS, JAUS, CBML/MSDL, etc.

Integration of simulations compliant with HLA standard can be done using our

own implementation of a HLA middleware: Simware RTI Pro. This

implementation, fully compliant with IEEE 1516.2000, is designed to improve the

real time performance in a HLA network.

Next picture shows how Simware IDE enables the rapid assembly, initialization,

testing, and execution of a system or system-of-systems from members of a pool

of reusable, interoperable software elements as TENA requirements are

demanding. Indeed, Simware is the only simulation platform in the market that

embraces the best agile&lean software practices to do it2.

1 LTI or Learning Tool Interoperability is the standard used in Simware to connect with web and mobile applications. More details at

http://www.simware.es/open.html

2 Go to http://www.simware.es/agile-simware.html to learn more about how to adopt Agility with Simware platform



http://www.simware.es/simware-web.html

http://www.simware.es/simware-recordplay.html

http://www.simware.es/simdeveloper.html

http://www.simware.es/simware-lvc.html

http://www.simware.es/simware-rti-pro.html

http://www.simware.es/open.html

http://www.simware.es/agile-simware.html



of 19



Figure 11 Rapid development of T&E exercises with Simware IDE




Understanding Simware architecture Date: July 2017

of 19



8. DEPLOYING THE SIMWARE BASED T&E NETWORK

In execution, Simware is deployed as any micro-services architecture, with a

simulation infrastructure deployed on top of the network3. Basic parameters of

the execution (distribution of the Simulation Entities, overall frequency of the

simulation, etc.) are defined in a XML configuration file that is part of the

runtime infrastructure layer. The middleware can be deployed on the network

using HLA or DDS protocols. Best performance is achieved with DDS and HLA as

the default exchange mechanism is only recommended for small scale

experiments, with many of the assets already compliant with this standard and

if you don’t need determinism.

Several sessions of the simulation can be running at the same time on the same

network. Simware supports the concept of Domains, that it is a logical scope

(or "address space") for the data-models definitions. In fact, when Simware tools

creates automatically the infrastructure (see details in former section), they are

creating an instance of the middleware for the specific simulation data-model.

Simulation domains are completely independent from each other. For two

Simware Entities to communicate with each other they must join the same

simulation Domain. This feature allows to run simultaneously several T&E

exercises in the same physical network.

Middleware will create messaging channels in each domain between the

publishers and subscribers of each type of data contained in the model.

Simware allows to optimize the performance of the publications and

subscriptions with specific service level agreements between the publishers &

subscribers and the middleware as defined in a QoS file. Simware provides a

default QoS file for all the publishers and subscribers but this XML file can be

modified by the user in any moment. This feature is very important for the case

of T&E exercises because it allows to change the network and connectivity

conditions between exercises without changing the entities.

3 A detailed description of the deployment architecture in Simware is included in the technical resource Understanding Simware

platform that can be found at www.simware.es/resources.html



http://www.simware.es/resources.html



of 19



In execution, Simware supports periodic and aperiodic operations. Periodic

operations need to be synchronized and execute periodically and this is done

by specific services that are part of the runtime infrastructure layer in Simware:

the Scheduler and the simulation engines. Aperiodic operations will work

asynchronously and will be probably event-driven.

One Scheduler service will be running on one machine of the simulation

domain. Command of the simulation can be made through ACS console

provided with the software or by any application that is connected to the

control data model.

Periodic simulations will run under the control of a SimEngine. One SimEngine

service will be running in each machine that it is running periodic simulations,

as physic-based simulation models.

Event based Entities can be also running in the simulation infrastructure. This kind

of aperiodic applications can use the synchronization services provided by the

Scheduler Service (common state-machine, common wall-clock) or run

asynchronously only coordinated by specific interactions defined in the

simulation data-model. For example, an interaction can be a Request_LOS that

is processed by an aperiodic Terrain simulation service. Any Entity requesting a

Line of Sight service will publish the interaction and the simulation service will

publish the LOS for the provided location.

Other entities can be connected to the middleware to consume or publish

data. This Entities can be integrated in Simware infrastructure through the C++

and Web APIs or by using a gateway.





of 19



9. EXAMPLE OF A VIRTUAL T&E NETWORK USING SIMWARE

Test and evaluation of system of systems demands the seamless connection of

heterogeneous assets, real and simulated. Real cases involve multiple

architectures and protocols.

One example is shown in next picture. This picture shows the high level

architecture of a real use case for Simware, the CITIUS Lab. This is an example

of a multi-architecture T&E solution that leverage the data-centric architecture

in Simware and its connectivity capabilities to integrate real and simulated

systems in a common virtual environment. In this case heterogeneous systems

are connected to a common simulation environment using different interfaces

as DDS, HLA, DIS, JAUS or MSDL/CBML.

In this solution, digital twins of the actual systems (unmanned platforms and its

main systems and equipment) allow to test the integration of the real control

stations of the unmanned systems with the naval combat system while is

operating a virtual version of the autonomous vehicle. Digital models of the

payload can be also connected to the actual autonomous vehicle.

Further details about this use case can be found at this page:

http://www.simware.es/use-case---citius.html



http://www.simware.es/use-case---citius.html



of 19



10. SUMMARY

A modern T&E solution requests the

rapid assembly of live systems, C4I

applications, sensors and digital

models & simulations in a common

synthetic battlespace. The best way

to achieve it is by using the Internet

of Simulations concept, that is the

extension of IoT to the M&S domain.

IoS provides a seamless connection

between all kind of things, user interfaces, processes, models and simulations.

Only Simware platform and its LSA architecture are right now providing the

capability to build a military T&E simulation network in accordance with IoS,

with a fully open, uncoupled and scalable architecture.

11. ABOUT SIMWARE SOLUTIONS

Simware Solutions is leading the introduction of Open platforms into the

Simulation & Training markets. Our platform, Simware, leverages the new

Layered Simulation Architecture or LSA to fulfill the requirements of the lead

users of the industry, which are demanding open architectures, better

interoperability and increasing economical returns for their investments in

simulation and training solutions.

Our platform is the first commercial product in the market supporting the

Internet of Simulations concept. IoS is about to embrace technologies as

internet, distributed systems, open platforms, cloud computing and service

oriented architectures for the development and deployment of open, net-

centric and interoperable simulations.

Simware is the only simulation platform in the market supporting Net-Centric

simulation without restrictions, enabling new business models for simulation as

the use of simulation as a Service (MSaaS) or the use of simulation platforms as

a service (SPaaS).

Simware is the only simulation platform in the market that is useful in all the

phases of the simulation based system engineering of industrial and consumer

products.



Virtual testing and evaluation with Simware and the Internet of Simulations

Technology

Transcript of Virtual testing and evaluation with Simware and the Internet of Simulations