Making DDS Really Real-Time with OpenFlow

Hyon-Young Choihyonchoi@cis.upenn.edu

Andrew L. Kingkingand@cis.upenn.edu

Insup Leelee@cis.upenn.edu

Department of Computer & Information ScienceUniversity of Pennsylvania

Philadelphia, USA

ABSTRACTAn increasing amount of distributed real-time systems andother critical infrastructure now rely on Data DistributionService (DDS) middleware for timely dissementation of databetween system nodes. While DDS has been designed specif-ically for use in distributed real-time systems and exposes anumber of QoS properties to programmers, DDS fails to lifttime fully into the programming abstraction. The reasonfor this is simple: DDS cannot directly control the underly-ing network to ensure that messages always arrive to theirdestination on time.

In this paper we describe a prototype system that uses theOpenFlow SDN protocol to automatically and transparentlylearn the QoS requirements of DDS participants. Based onthe QoS requirements, our system will manipulate the low-level packet forwarding rules in the network to ensure thatthe QoS requirements are always satisfied.

We use real OpenFlow hardware to evaluate how well ourprototype is able to manage network contention and guar-antee the requested QoS. Additionally, we evaluate how wellthe reliability and resilience features of a real DDS imple-mentation is able to compensate for network contention onan unmanaged (i.e., normal) ethernet. To the best of ourknowledge, this is the first evaluation of performance DDSunder extreme network contention conditions.

CCS Concepts•Computer systems organization ! Real-time oper-

ating systems; Real-time system specification; Peer-to-peer architectures; Reconfigurable computing; Maintain-ability and maintenance; •Networks ! Network experi-mentation;

KeywordsReal-Time, Distributed Systems, Publish-Subscribe, Soft-ware Defined Networking, QoS Management

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permissionand/or a fee. Request permissions from permissions@acm.org.

EMSOFT’16, October 01-07 2016, Pittsburgh, PA, USAc� 2016 Copyright held by the owner/author(s). Publication rights licensed to ACM.

ISBN 978-1-4503-4485-2/16/10. . . $15.00

DOI: http://dx.doi.org/10.1145/2968478.2968479

1. INTRODUCTIONThe development of distributed real-time software systems

is a complex and involved process.The distribution of systemfunctions across multiple hosts means programmers mustcontend with a plethora of issues related to logical concur-rency and low-level networking details. In most real-worldsystems integration contexts, systems developers are alsochallenged by interoperability, i.e., how to ensure that com-ponents produced by di↵erent vendors can understand eachother and communicate e↵ectively. Additionally, for systemswith real-time requirements, programmers and systems en-gineers must also ensure that the overall system exhibits thecorrect timing behavior.

Over the past two decades there has been significant e↵ortby both the academic community and industry to addressmany of the distributed real-time systems development chal-lenges via middleware [15, 4, 3]. The goal is to have the mid-dleware provide a simplified programming abstraction thathides important, yet tedious and complex, implementationdetails such as data transport, service discovery/routing, se-rialization/deserialization and many others.

Recently, there has been increasing interest in middlewareimplementing the Data Distribution Service (DDS) ObjectManagement Group (OMG) standard [11]. There are severalhigh-quality implementations of the standard. Furthermore,a subset of these implementations have been rigorously ver-ified and include certification packagages enabling their usein systems that require DO-178C Level A, IEC 60601, orISO 26262 certification.

The quality of DDS middleware has lead to many largescale distributed real-time software systems that depend onit such as the Littoral Combat Ship [3], The Grand CouleeDam [5], and Duke Energy’s next generation smart grid [16].The increasing popularity of DDS cannot be overstated: Onemajor DDS vendor now claims that the total value of sys-tem designs that depends on their implementation of DDSexceeds “1 trillion dollars” [7].

DDS provides a“data-centric”publish-subscribe program-ming abstraction designed for use in real-time systems. Dataproducers (i.e., publishers) write updates to topics and dataconsumers (subscribers) specify which topics they want tomonitor and receive updates from. DDS lets the program-mer attach Quality of Service (QoS) settings to publishers,subscribers, and topics. These settings define how the DDSmiddleware responds to faults (e.g., connectivity loss) andlets the programmer specify the distributed timing require-ments of the system (e.g., the DEADLINE parameter, whichspecifies how often a subscriber should receive updates from

a publisher).While the DDS standard provides a rich set of real-time

centric QoS settings, DDS middleware does not fully liftsystem timing into the programming abstraction. Indeed,bringing timing fully into the programming abstraction isdi�cult because the timing behavior of a software systemonly emerges when you map the software onto a particularplatform (i.e., processor, operating system, and networkingfabric). In the context of a critical distributed hard real-timesystem, special real-time networking hardware is often usedand the systems integrator must devote significant e↵ortprovisioning and configuring that hardware to ensure therequired timing behavior. The configuration process usuallyinvolves setting priorities or transmission schedules for eachtime critical flow. Sometimes, if the underlying network willbe shared by more than one logically independent applica-tion, tra�c policers and/or special partitioning features aresetup to keep the di↵erent applications isolated. These net-work configurations are usually static and setup o✏ine: Ifone wants to add a new node to the network (e.g., a sensoror actuator) the system must be brought down, manuallyreconfigured, and then brought back up. The process ofnetwork resource configuration can be complex, costly, anderror prone.

Recent advances in Software Defined Networking (SDN)have the potential to enable the enforcement of a distributedsystem’s middleware-specified timing properties automati-cally and transparently in the network hardware. In theory,a programmer could specify distributed timing constraints,such as end-to-end latency requirements, to the middlewareitself. Then the middleware could then leverage SDN pro-tocols such as OpenFlow [10] to automatically and dynami-cally reconfigure the underlying network to enforce the spec-ified timing constraints.

Using SDN to lift timing behavior into the programmingabstraction o↵ers a number of potential benefits. First, itmeans systems integrators would no longer need to manu-ally provision network resources o✏ine. Second, it allowsfor the use of general-purpose SDN hardware in real-timeapplications. Third, it allows for online dynamic networkreconfiguration thereby enabling use-cases with real-time re-quirements but whose complete configuration is not knowna priori (e.g., the Industrial Internet of Things (IIoT) [1, 14]and plug & play medical systems [12]).

The aims of this paper are two-fold. First, we propose anSDN controller that automatically enforces requested QoSdirectly in the network and prevents network overloads fromimpacting the DDS tra�c. A key design feature of our con-troller is that it does not require any modification to exist-ing DDS implementations and in theory could be deployedin existing systems. This feature is important because itmeans certified DDS implementations would not need to gothrough expensive recertification to take advantage of oursystem. Second, we want to evaluate how a real DDS imple-mentation is able to compensate for less than ideal networkconditions such as overloads. Because an increasing amountof critical infrastructure is depending on DDS for correctfunction, it is important to understand how the middlewarebehaves under these conditions. As far as we know, up untilthis point, no such study has been performed or publicallyreleased.

This paper is organized as follows: In Section 2 we pro-vide a overview of DDS middleware. Section 3 gives a brief

background of OpenFlow. Section 4 describes new proposedQoS parameters and gives an overview of the architectureand function of our SDN controller. Section 5 contains theexperimental evaluation. In Section 6 we survey and discussrelated work. We conclude and propose areas for future workin Section 7.

2. DDS BACKGROUNDDDS provides a data-centric publish-subscribe program-

ming abstraction [11]. The DDS abstraction exposes a num-ber of features programmers can use to disseminate dataacross a distributed system at di↵erent levels of granularity.DDS also provides a rich set of QoS settings that can be usedto both control and specify the non-functional properties ofthe system. A complete description of the DDS program-ming abstraction is beyond the scope of this paper. Instead,in this section we give a functional overview of the DDSprogramming abstraction. We describe, at a high-level, thedi↵erent abstraction entities, their roles, and how they aretypically implemented. We give special attention to the dif-ferent QoS settings that can either a↵ect or specify desiredsystem timing.

2.1 Data Centric Programming AbstractionFigure 2 illustrates the basic DDS programming abstrac-

tion. Any software that wants to produce and share datauses a DataWriter object. The DataWriter is associatedwith a Topic. If another program (which could be on a dif-ferent host) wants that data, it uses a DataReader that issubscribed to that topic. We say that DDS is data-centric be-cause Topics have structured data types and the data typingused in DDS lets us view Topics in a way that is intuitivelysimilar to a table in a relational DBMS: DataWriter objectsare a portal programmers can use to update the data storedin the table, and DataReader objects provide a view of thedata in the table. Indeed, the DDS specification lets pro-grammers use SQL-like queries with DataReaders to extractthe specific data they desire.

Figure 1 shows how the top-level programming abstrac-tion relates to intermediate abstractions and the underly-ing implementation. At the top level are the DataWriters,DataReaders, and Topics which embody the datacentric por-tion of the abstraction. In DDS, DataWriters are bound toPublishers and DataReaders are bound to Subscribers. Thepublishers and subscribers sit on top of modules that imple-ment the Real-Time Publish Subscribe Protocol or RTPS.The RTPS module is split into two sub-modules, a platformindependent module and a platform specific module. ThePlatform Independant Module (PIM) defines the types ofmessages and state-machines that are used by the RTPS.The Platform Specific Module (PSM) maps the logical pro-tocol elements onto a specific networking implementation(e.g., IP/UDP, CAN Bus, etc).

2.1.1 Quality of Service ParametersDDS supports over two dozen QoS policies that can be

specified for the di↵erent entities. The QoS policies af-fect how the middleware disseminates data from publishersto subscribers. Furthermore, the middleware automaticallychecks the consistency of the QoS settings between entitiesinvolved in a publish/subscribe relationship and notifies theapplication of any violation. The QoS policies supported byDDS a↵ect various non-functional properties of the publish-

ParticipantParticipant

DataWriter DataReaderTopic

Publisher Subscriber

RTPS PIM

IP/UDP PSM

RTPS PIM

IP/UDP PSM

IP Network

QoSType Name

QoS QoS

QoS QoS

Prog

ram

min

g Ab

stra

ctio

n Implem

entation / Realization

Figure 1: DDS programming abstraction entities and implementation.

subscribr system including data availability, delivery andtiming. We will not cover every aspect of DDS QoS here.Instead we will only focus on the parameters that typicallya↵ect timing (either directly or indirectly).

• DEADLINE(number) - Defines the maximum separa-tion between consecutive updates to a topic. DataWrit-ers must not wait longer than the DEADLINE betweenconsecutive writes. DataReaders will generate an ex-ception if the deadline is exceeded between updates(i.e., if a DataWriter is too slow or the underlyingnetwork delays delivery of an update).

• LATENCY_BUDGET(number) - Hint to the middlewarespecifying tolerable update delivery delay. The latencybudget is used by the underlying middleware to decidewhether it can batch updates together in order to op-timize network bandwidth.

• BEST_EFFORT(boolean) - If true, the middleware won’tattempt to resend lost updates. Publishers will sim-ply transmit topic updates as fast as possible onto thenetwork.

• RELIABLE(boolean) - Specifies whether a reliable trans-port must be used to deliver updates. Messages lostdue to network contention or faults will eventually berecovered by the subscriber.

• HISTORY_DEPTH(number) - Defines how large the up-date cache must be in a DataReader or DataWriter.For DataWriters the history depth indicates how manyupdates it will keep available for recovery in RELIABLEsessions. For DataReaders it is simply the number ofprevious updates the DataReader can make availableto the application.

• DESTINATION_ORDER - Specifies whether the subscriberDataReader must present topic updates to the applica-tion in the order they were written to the DataReader.

• User QoS - DDS also allows users to specify their ownQoS parameters that can be processed by their appli-cation. User QoS lets users attach arbitrary key/valuepairs to DataReader/DataWriters. These key/valuesare exchanged during discovery but will not a↵ect thebehavior of the middleware. Instead, they can be re-covered and used by the application for whatever pur-pose as intended by the programmer.

2.2 OperationThe operation of DDS can be divided into three main

phases: 1) Discovery - where the di↵erent DDS participantsfind each other on the network, 2) Matching - where thediscovered participants determine if they should engage ina publish-subscribe relatonship, and 3) Data Distribution- where data is disseminated from the publishers to sub-scribers that have matched. The particulars of how DDSoperates depends on the type of networking technology be-ing used and the Platform Specific Module (PSM) imple-mention of the RTPS for that network. Here we give a high-level overview of how these di↵erent phases are implementedin the IP/UDP PSM that is used when DDS is operated ontypical IP networks.

2.2.1 DiscoveryThe discovery phase is itself divided into two sub-phases:

Participant discovery and endpoint discovery. The aim ofparticipant discovery is to discover each node on the net-work that is running DDS. Once the DDS participants havebeen discovered, each participant will attempt to discoverthe end-points (i.e., DataWriters & DataReaders) hostedon the other participants and what topics those end-pointsare publishing or subscribing to.

The IP/UDP PSM implements participant discovery us-ing the Simple Participant Discovery Protocol (SPDP). TheSPDP works by IP multicasting: Each participant periodi-cally transmits a discovery message to a pre-defined multi-cast address. The discovery message contains the informa-tion needed by other participants to uniquely identify theoriginator of the message. Participants also listen for mul-ticasts to that same address. When a participant detectsanother participant, the information contained in the dis-covery message is stored in a local “participant database”.

After a participant discovers another participant it will

DataWriter DataReaderBoilerPressure

QoS:LATENCY_BUDGET = 15DEADLINE = 100HISTORY = 20…

QoS:LATENCY_BUDGET = 30DEADLINE = 100HISTORY = 10…

QoS:DEADLINE = 200HISTORY = 3…

Figure 2: Publish-subscribe relationship with consistentQoS settings.

initiate end-point discovery using the Simple End-point Dis-covery Protocol (SEDP). SEDP is a reliable (i.e., it will re-send lost messages) point-to-point protocol that allows oneparticipant to learn what the other participant’s DataRead-ers/DataWriters are, the topics/data-types associated withthose DataReaders/DataWriters, and their respective QoSsettings. Additionally, participants learn a “locator list”matching topics to IP addresses. Typically, each topic willbe associated with its own multicast address / IP port com-bination to facilitate e�cient dissemintation of topic updatesover the network. The participants will store the informa-tion learned about remote end-points in a local end-pointcache.

2.2.2 MatchingAfter the end-points have been discovered the matching

process will begin. Subscribers will examine their local end-point cache to see if they have discovered any end-pointsthat are publishing to the topics they are subscribing to.If the subcriber find an end-point that is publishing to atopic they subscribe to, the subscriber will compare its ownQoS requirements to the QoS o↵ered by the publishers. Ifthe QoS settings are consistent (i.e., the publisher’s QoSguarantee is at least as strong as the subscriber’s require-ments) then the match is successful and the subscriber willstart receiving topic updates on behalf of its DataReaders.Since multicast locators are typically used, the subscriberwill start listening on the appropriate address for updates.

2.2.3 Data DistributionAt a high-level, IP/UDP data distribution is fairly sim-

ple and works by having the publishers serialize the updatedata, encapsulate that data with DDS specific information(e.g., sequence number) and then transmit the encapsulateddata as a UDP packet to the multicast address associatedwith the topic in question. The subscribers subscribed to thetopic will be listening for packets on that multicast address.When a subscriber receives a new packet they will deseri-alize the update and present it to the application via theassociated DataReader. While basic operation is fairly sim-ple, the details of how the RTPS and IP/UDP PSM handlesdata distribution changes based on the QoS settings used.

• BEST_EFFORT = true - If BEST_EFFORT QoS is activethen the publisher will “fire and forget” topic updates.Updates lost in transit will not be recovered.

• RELIABLE = true - The IP/UDP PSM will use a re-liable multicast procotol to recover lost update mes-sages. Subscribers will keep track of each received up-date’s sequence number. NACK messages are sent tothe publisher for each sequence number missed andthe publisher will retransmit the updates (that is hascached) for the missing sequence numbers.

• HISTORY_DEPTH = n. When reliable transport is en-abled, the publisher can only resend lost updates thatare present in its local cache. If the publisher no longerhas the update specified by a NACK it will notify thesubscriber that the data is no longer available.

• DESTINATION_ORDER = true. If a DataReader is setto enforce ordering it will not present the most re-cent update to the application until all previous up-dates have been presented. If intermediate updates

have been lost due to network faults or contention, theDataReader will wait to present recent updates untileither the missing updates have been recovered andpresented, or the publisher has indicated it no longerhas the missing updates in its update cache.

3. OPENFLOW BACKGROUNDOpenFlow [10] is a protocol designed to enable Software

Defined Networking (SDN). SDN separates the data plane(where data packets are switched, routed and otherwise moved)and the control plane, which decides how to configure thedata plane based on high-level routing or switching poli-cies. Traditional networking equipment contains both thecontrol and data planes. For example, a typical COTS eth-ernet switch implements the data-plane in hardware using aspecialized packet-switching ASIC while the firmware run-ning on the switch’s general purpose CPU will configurethe switching ASIC according to whatever policy the switchmanufacturer has defined.

SDN protocols like OpenFlow enable the reconfigurationof the switch hardware ASIC from a server process runningon a remote machine (called the OpenFlow controller). Thiscapability makes it posible to write “controller applications”using a language like Java or C++ that learn the topology ofthe network (i.e., learn what ports on what switches di↵er-ent hosts are connected to) and then e↵ect complex routing,forwarding and QoS strategies.

We now describe the operation of an OpenFlow network.An OpenFlow network consists of OpenFlow switches andOpenFlow controllers (e.g., Floodlight, Ryu, etc.). An Open-Flow hardware switch is a Layer 2/3 Ethernet switch thatmaintains a table of flow entries and actions.

The flow table associates each flow with a set of actionsthat indicate to the switch how to handle packets matchingthe flow. The OpenFlow standard defines a number ofdi↵erent possible actions a switch may implement includingenqueue (place the packet on a specific egress queue), limit(apply a rate-limiter) and drop (simply drop the packet).

When a switch receives a packet on one of its interfaces,it tries to find a matching flow entry in its flow-table. If amatching entry is found, the associated action set is applied.If the packet does not match an existing entry the switchperforms a Packet-in. When the switch performs a Packet-in, it sends a message to the OpenFlow controller (a piecesoftware running on a server in the network). The messagecontains the packet that failed to match as well as meta-information about the packet such as on which switch thematch failure occurred, the physical port of the switch thatthe packet arrived on, and other statistics. The controllercan use the meta-information and the information in thefailed packet itself to decide on a course of action. Thecontroller can then either modify one or more switch’s flowtables with a new rules and/or send the packet back to aswitch with instructions on how to deal with just that packet(called Packet-out).

4. OVERVIEWOur controller works by using the OpenFlow protocol to

transparently insert itself into the DDS discovery process.It uses the OpenFlow protocol to dynamically learn whereDDS nodes exist on the network and what each nodes spec-ified QoS is. If two nodes want to engage in a publish-

subscribe relationship, the controller extracts the specifiedend-to-end QoS by intercepting and deserializing the discov-ery messages. The controller will then attempt to derive anetwork configuration and resource reservation that satisfiesthe specified QoS. If the new configuration will guarantee theQoS the controller will let the discovery process proceed andthen commit the new configuration to the network so DDSdata tra�c can flow from publisher to subscriber.

4.1 Proposed QoS AdditionsWhile our system does not require any modifications to

the DDS middleware itself, it does need information thatcan’t be easily derived from the existing standardized QoSparameters. Here we propose two new QoS parameters thatwill enable us to support automatic timing guarantees. Whileour current system requires that the application developersupply these parameters to the middleware via the User QoSfeature, we hope that these (or equivalent) parameters willeventually be considered for inclusion in the DDS standard.

The first QoS we propose is called MINIMUM_SEPARATION.MINIMUM_SEPARATION applies to Publishers and defines thesmallest allowed interval between the network transmissionsrelated to two consecutive updates to a topic. In this paperwe use minseptp to denote the MINIMUM_SEPARATION appliedto publisher p for topic t. Formally, we define minseptp asfollows:

Definition 1. Publisher minimum separation. Let tiand ti+1 be any two moments when the publisher p startstransmitting the data for topic t update i and i+ 1 respec-tively. Then it is always the case that minseptp ti+1 � ti.

MINIMUM_SEPARATION is essential for our ability to doschedulability analysis: It precisely defines how often a pub-lisher can transmit a data on the network. When com-bined with the maximum data-size for an update to a givendata-type, MINIMUM_SEPARATION lets us derive how muchload the publisher can induce on the network which is re-quired for schedulability analysis and the ability to guar-antee QoS. The second proposed Qos parameter concernsend-to-end latency. While the DDS standard already has theLATENCY_BUDGET QoS, its semantics is purposely not well de-fined. Furthermore, LATENCY_BUDGET is intended to apply topublishers as a hint how to batch topic updates for networktransmission. We propose a new latency QoS in order toprevent confusion with DDS’ LATENCY_BUDGET. Our new pro-posed parameter for end-to-end latency, called E2E_LATENCY,defines an upper bound on the amount of time it can takean update to traverse the network from publisher to sub-scriber. Unlike DDS’ LATENCY_BUDGET, E2E_LATENCY is in-tended to be specified by Subscribers. Furthermore, the se-mantics E2E_LATENCY require the underlying network infras-tructure to guarantee that the specified latency always besatisfied. In this paper we use lts to denote the E2E_LATENCYspecified by subscriber s for topic t:

Definition 2. End-to-end latency. Let tpi be the mo-ment publisher p transmits the first bit of update i on thenetwork and tsi be the moment subscriber s receives the lastbit of update i. Then for any i, it is always the case thattsi � tpi lts.

4.2 Operation & ArchitectureFigure 3a shows the functional software architecture of

our SDN controller and Figure 3b shows how DDS discov-ery & data tra�c flows in our solution. The controller isimplemented in Java as a “controller” application runningon top of the Floodlight controller. Floodlight o↵ers basicfunctionality including:

• TopologyService - Maintains the topology of the net-work. The topology service periodically causes theswitches under the control of the controller to sendLink Layer Discovery Packets (LLDP) out of their ports.Changes in the physical topology of the network (e.g.,due to cable connect/disconnect) will be detected bythe topology service, which will then notify the otherFloodlight modules and controller applications.

• Forwarding/Routing - These two modules will learnthe MAC address of all network connected devices andwill push flow rules to ensure that non-DDS tra�c cantransit the network as if it was a normal (non-SDN)ethernet. We’ve modified these modules from the onesavailable in the normal Floodlight distribution to en-sure that non-DDS tra�c will always be queued at alower priority than the DDS tra�c.

• DeviceManager - Maintains database of the hosts con-nected to the network including their MAC/IP ad-dresses and what port of what switch the host is pluggedinto.

In addition to the above, our controller runs four moremodules specifically designed guarantee the QoS of DDStra�c:

• DDSDiscoveryInterceptor - Handles Packet-In eventsfor all DDS discovery (i.e., SPDP or SEDP) tra�c.This module eavesdrops on the discovery procotol andlearns what network devices are DDS participants,whether two participants have end-points that wantto engage in a publish-subscribe relationship, and whatthe requested QoS for that relationship is.

• DDSFlowScheduler - Does admission control for publish-subscribe relationships. Generates a network configu-ration (i.e., flow rules) that potentially guarantees theQoS of all publish-susbcribe flows. Uses schedulabilityanalysis to determine if the new network configurationis feasible. If so uses the NetworkModel to safely up-date the flow rules in the switches.

• NetworkModel - Stores dynamic information about thestate of the network and resource reservations for DDSclients. It tracks which nodes are DDS participants,which participants are engaged in a publish-subscriberelationship, the QoS of those relationships, and theflow rules installed into each switch to support thoserelationships.

• SwitchModelDB - Stores static information about theperformance characteristics of each switch in the net-work. This information is used by the flow schedulerto generate new network configurations and to per-form feasibility testing of those configurations. Theinformation includes, for each model of switch usedin the network, multiplexing latency, the number of

DDSFlowSchedulerAdmissioncontrol&

DDSflowconfig.generation.

DDSDiscoveryInterceptorInterceptsDDSdiscoverytraffic

SwitchModelDBInfore:switchqueues,scheduling,rate-limits

NetworkModelAbstractscurrentnetworkconfig.

ForwardingRouting

TopologyServiceDeviceManager

FloodlightController

Sequence of flow rules updates (add,del, or mod)

Notify if admitted

Ask for admission

Packet-Out

Packet-Out,Flow-Mod,Continue

Flow-Mod

Packet-In

Packet-In

Controller App Components

Floodlight Modules

(a) Software architecture of the SDN controller.

DDS Data

DDS Discovery Traffic.Openflow Rules

Switch Switch SwitchDDSPublisher

DDSSubscriber

SDN Controller

………… …………

(b) Routing of DDS data, DDS discovery, and OpenFlow flow-modification messages.

Figure 3: SDN controller design & operation.

egress queues per port (and the depth of the queues),the intra- and inter-queue packet scheduling disciplineused (e.g., FIFO, WFQ, FP, etc.), and the precision ofthe ingress rate-limiters.

Using these modules the controller application works asfollows. When DDS clients start participant discovery viathe SPDP they will multicast discovery packets. We ensurethat the flowtable of each switch has no entry that can matchagainst SPDP tra�c. This causes a switch that receivesthe SPDP tra�c to forward it to the controller via Packet-In. On the controller side, Packet-In’s related to SPDP orSEDP tra�c is forwarded to the DDSInterceptor module.The DDSInterceptor deserializes the SDPD messages andlearns the IP address and network location (i.e., the switchand port) of the DDS participant. This information is thencached in the NetworkModel. The interceptor then Packet-outs the SPDP packet to the other DDS participants.

Next, once the participants have discovered each other,they will initiate end-point discovery with the SEDP. Again,since the SEDP tra�c will not match any rule in any switch’sflow table the SEDP tra�c will be forwarded to the con-troller via Packet-In. The controller uses the SEDP tra�c tolearn which DDS participants want to engage in a publish-subscribe relationship and the requested QoS settings forthat relationship. If both MINIMUM_SEPARATION (publisher)and E2E_LATENCY (subscriber) has been specified then the in-terceptor will initiate admission control with the FlowSched-uler.

The FlowScheduler will attempt to derive a network con-figuration that guarantees the requested QoS. The mathe-matical details of how such a scheduler could work are be-yond the scope of this paper (See [8] or [9] for two di↵erentapproaches. We follow the approach of [8]). Instead we willgive a high-level description. First the FlowScheduler usesthe publisher’s MINIMUM_SEPARATION and the topic’s datatype to infer a worst-case arrival pattern of bits that the flowof updates from the publisher can induce onto the network.Then, the FlowScheduler uses the information stored in theNetworkModel to find a path for the flow across the networkthat satisfies the latency requirement. The FlowSchedulermay choose to prioritize the new flow di↵erently at eachhop to ensure that each real-time DDS flow (both new andold) will always satisfy its QoS. After it has generated a newconfiguration, the FlowScheduler will test the configuration

for schedulability.If any flow admitted to the system can miss its speci-

fied QoS the admission process for the new flow fails andthe FlowScheduler will instruct the DDSInterceptor to blockthe matching process between the publisher and subscriberin question. If the new configuration guarantees the flow’sQoS the flow scheduler will give an ordered list of“flow mod”commands to the NetworkModel. The NetworkModel willcache the new flow configuration and then commit it to thenetwork in the order specified (The ordering generated bythe FlowScheduler ensures that QoS of existing flows willnot be violated during the reconfiguration process). Afterthe new configuration has been comitted to the network theFlowScheduler will instruct the DDSInterceptor to let theSEDP proceed so the publisher and subcriber can match.Since flow rules matching the new publish-subscribe floware now present in the switches, all publish subscribe traf-fic between the new end-points can flow freely and will beprocessed at line rate.

5. EXPERIMENTAL EVALUATIONOur experiments have two goals. First, we want to un-

derstand how a real DDS implementation compensates fornetwork overloads. In particular, we want to evaluate hownetwork overload conditions a↵ect the end-to-end timing ofthe publish-subscribe relationships and then how well theDDS reliability and resilience features (e.g., RELIABLE, HIS-TORY_DEPTH, and DESITINATION_ORDER QoS) help the ap-plication recover from message losses incurred during theoverload. The second goal is to evaluate how well our SDNcontroller is able to protect DDS tra�c from interferencefrom network overloads. We will compare the timing per-formance of the system when our controller is enabled towhen it is not and when the network is and isn’t overloaded.

5.1 SetupFigure 4 shows the experimental setup. We built a net-

work comprised of a single Pica8 P-3297 Gigabit top of rackswitch, three Linux computers (one for the publisher, onefor the subscriber, and one to run the SDN controller) anda Linux desktop to act as a tra�c generator. In order to en-sure that we can overload the queuing capacity of the switchport connected to the subscriber host the tra�c generatoris connected to the switch with two 1Gbps links.

Pica8 Switch(OVS Mode)

DDSPublisher(RT Linux)

1

2 4

3C

DDSSubscriber(RT Linux)

TrafficGenerator

SDNController

1Gbps

1Gbps 1Gbps

1Gbps

Figure 4: Experimental setup.

The publisher/subscriber computers were desktop machineswith Intel Core i7-3770 (publisher) or i7-4770 (subscriber)processors, 8GB (publisher) or 16GB (subscriber) of RAMand configured with the fully preemptible (i.e., RT_PREEMPT)real-time Linux kernel version 4.4.4-rt11.

We wrote a custom application in real-time (RTSJ) Java(running on IBM’s WebSphereRT real-time JVM) that pub-lishes an update to topic TestTopic every 5ms. Likewise,we wrote a subscriber application that uses DDS to sub-scribe to TestTopic. We chose Real-Time Java because ito↵ers a good combination of programmer productivity, pre-dictability, and lets us avoid certain annoying issues thatcan manifest when unmanaged programming languages areused (e.g., heap fragmentation). Both the subscriber andpublisher have DEADLINE QoS set to 5ms. The publisher’sMIN_SEPARATION is also set to set to 5ms and the E2E_LATENCYof the subscriber is set to 1ms. The DDS implementationused is Real-Time Innovation Inc.’s Connext DDS [7]

Each time the subscriber got an update it would recordwhen it got the update, the sequence number of the update,and a count of the cummulative updates. The combina-tion of real-time Java, IBM’s WebSphereRT, and real-timeLinux a↵orded us fairly precise control over when the pub-lisher actually published an update to a topic, and whenthe subscriber activated in response to receiving an update:Observed dispatch jitters (i.e., variation in when scheduledevents, such as topic writes, happen in the Java application)were always 100µs. Latency through the network card,kernel networking stack, JVM and middleware were alwaysless than 1ms total.

The tra�c generator host was equipped pktgen packetgenerator program. When activated, it generated a floodof UDP packets towards a the subscriber’s IP address in aspecific pattern: The generator would cycle flooding for 1second, then stop for one second. The goal of this patternis to help us see the e↵ects of transient overloads. Usingboth network links the tra�c generator could consistentlyflood the network at 2,000Mb/s, more than enough to over-whelm the capacity of the subscriber’s 1000Mb/s connectionto the switch. We ran our test publish-subcribe applica-tion a number of times with a variety of QoS settings andwith/without our SDN controller. For each run we ran ourpublish-subscribe system for 25 seconds. Over the course ofthese runs we captured two categories of data-sets. The firstcategory is cummulative updates received vs time. The otheris Normalized Latency vs. Seq. Number.

5.1.1 Cummulative Updates vs. TimeThe cummulative updates vs. time datasets give the num-

ber of updates to TestTopic the subscriber’s DataReader re-ceived and processed by time t. We ran 11 categories of thisexperiment, 6 with BEST_EFFORT QoS and 5 with RELIABLE

QoS. In all categories DESTINATION_ORDER QoS is active:

• BEST_EFFORT without contention or SDN, DataSize =100 bytes: This variations is to establish a baseline of“ideal” behavior. As DEADLINE = 5ms we expect thesubscriber to receive updates at a uniform rate of 200per second because there is no network contention.

• BEST_EFFORT without SDN but with periodic contention,DataSize = n bytes with n 2 {100, 500, 1000, 1400}:Captures the performance degradation due to networkcontention.

• BEST_EFFORT with SDN, periodic contention andDataSize = 1400. Captures the ability of the SDNcontroller to e↵ectively manage the contention and pre-serve the specified timing behavior of the publish sub-scribe system.

• RELIABLE without SDN but with periodic contention,HISTORY_DEPTH = n bytes with n 2 {1, 50, 100,1}:Captures the ability of the reliable transport to com-pensate for network packet drops.

• RELIABLE with contention and SDN, HISTORY_DEPTH= 1: This variations captures the performance of thesystem when the SDN control is enforcing QoS and thereliable multicast transport is being used.

Additionally, for each category subject to tra�c loadsfrom the tra�c generator we ran 15 sub-variations wherewe varied the size of the generated packets from 100 - 1500bytes in steps of 100 bytes while keeping the overall tra�crate constant (in e↵ect varying the packets per second).

5.1.2 Normalized Latency vs. Seq. NumberThe normalized latency vs. seq. number datasets give

when a particular update was received vs. when it was ex-pected:

Definition 3. Expected Update Time - Let i denote anupdate. Let t0 be the time the first update to the topic isreceived. Then the expected update time for update i, e (i)is calculated with the following equation:

e (i) = (i⇥ 5ms) + t0 (1)

Definition 4. Normalized Latency - Let ti be the time up-date i is actually received. Then the normalized latency ofan update i, nl (i) is the di↵erence of the actual update timefrom the expected time:

nl (i) = ti � e (i) (2)

Normalized latency helps us understand the e↵ects of net-work contention on how the application perceives the globaldata space with respect to time. As with the cummulativeupdates vs. time data sets we capture data for a numberof QoS variations including BEST_EFFORT & RELIABLE trans-ports, di↵erent HISTORY_DEPTH and with/without SDN sup-port.

Observe that, even in the ideal case, we don’t expect thenormalized latency to be 0 due to processing and networkjitters. Also, when SDN support is enabled, we expect smallvariations in the normalized latency because E2E_LATENCY= 1ms (update i could have an actual latency close to 0 butupdate i+ 1 could have an actual latency of 1ms).

0

1000

2000

3000

4000

5000

0 5 10 15 20 25

Rec

eive

d C

ount

Time(s)

Best-Effort(No-load) w/o SDNBest-Effort(Data-Size=100) w/o SDNBest-Effort(Data-Size=500) w/o SDN

Best-Effort(Data-Size=1000) w/o SDNBest-Effort(Data-Size=1400) w/o SDNBest-Effort(Data-Size=1400) w/ SDN

(a) BEST_EFFORT with 1400-byte background tra�c packets.

0

1000

2000

3000

4000

5000

0 5 10 15 20 25

Rec

eive

d C

ount

Time(s)

Best-Effort(No-load) w/o SDNReliable(Depth=1) w/o SDN

Reliable(Depth=50) w/o SDNReliable(Depth=100) w/o SDN

Reliable(Depth=∞) w/o SDNReliable(Depth=1) w/ SDN

(b) RELIABLE with 1400-byte background tra�c packets.

0

1000

2000

3000

4000

5000

0 5 10 15 20 25

Rec

eive

d C

ount

Time(s)

Best-Effort(No-load) w/o SDNBest-Effort(Data-Size=100) w/o SDNBest-Effort(Data-Size=500) w/o SDN

Best-Effort(Data-Size=1000) w/o SDNBest-Effort(Data-Size=1400) w/o SDNBest-Effort(Data-Size=1400) w/ SDN

(c) BEST_EFFORT with 100-byte background tra�c packets.

0

1000

2000

3000

4000

5000

0 5 10 15 20 25

Rec

eive

d C

ount

Time(s)

Best-Effort(No-load) w/o SDNReliable(Depth=1) w/o SDN

Reliable(Depth=50) w/o SDNReliable(Depth=100) w/o SDN

Reliable(Depth=∞) w/o SDNReliable(Depth=1) w/ SDN

(d) RELIABLE with 100-byte background tra�c packets.

Figure 5: Cummulative updates vs. time.

5.2 Results & DiscussionFigure 5 shows the cummulative updates vs. time dataset.

The results for background tra�c packet sizes � 200 isthe same so we only show the results for packet sizes 1400(Figures 5a & 5b) and 100 (Figure 5c & 5d). For bothBEST_EFFORT (Figure 5a) and RELIABLE (Figure 5b) QoSthe e↵ects of periodic network contention are readily ap-parent when SDN support is disabled. However, the e↵ectsmanifest di↵erently depending on whether BEST_EFFORT orRELIABLE is used. When there is no contention the be-havior of the system very closely matches the idealized ex-pected behavior: the slope of the line indicates that the sub-scriber is uniformly receiving updates every ⇠ 5ms. WhenBEST_EFFORT QoS is used periodic networking contentioncauses both updates to be dropped in the network and theoverall update rate to be depressed (causing DEADLINE QoSviolations in the DataReader).

The periodic pattern of the contention is clearly visible inFigure 5a: For the lines representing the data sets withoutSDN support, their slope is depressed for the 1 second du-rations the tra�c generator is active and the slope returnsto the idealized rate once the tra�c generator is o↵. Thedi↵erence between the line “idealized” line (representing therun where there is no load) and the other lines is numberof updates that were lost due to network overloads. As wecan see, the the data-size of the update does not a↵ect theimpact of the network contention. When the SDN controlleris managing the tra�c flows the tra�c generator causes noa↵ect on the rate the subscriber receives updates. Indeed,the line for “Best-E↵ort(Data-Size=1400) w/ SDN” exactly

tracks the line for when there is no contention.A system running with RELIABLE QoS reacts di↵erently

to network contention (see Figure 5b). Instead of only ex-periencing an update rate slowdown when the contention isactive (i.e., for 1 second durations) like with BEST_EFFORTQoS, updates are prevented from being processed by theDataReader for over 8 seconds with RELIABLE QoS. Whyis this? There are two factors at play (confirmed via codeinspections and discussions with the middleware vendor).First, because DESTINATION_ORDER = true the middlewarewon’t manifest updates to DataReader until all previous up-dates have been manifested: If one update is dropped, allsubsequent updates that are received are held back from theapplication by the middleware until the subscriber receivesthe missing update in e↵ect. Second, since the publishermust retransmit the missed updates, network contention isactually increased, making it more likely for further updatesto drop.

Though RELIABLE QoS can exaggerate the duration thatupdates are missed, if the publisher’s HISTORY_DEPTH is largeenough the middleware is able to exploit network idle timesto recover the missing data. We see this behavior with theline for“Reliable(Depth=1) w/o SDN”: No updates are pro-cessed by the DataReader once the tra�c generator activatesat t = 1 second until t = 8 seconds, at which point all themissing updates have been recovered and the received up-date count jumps to where it should be if no updates hadbeen lost. Again, like with BEST_EFFORT QoS, the SDN con-troller manages the tra�c flows to ensure idealized behaviorwhen RELIABLE QoS is used.

However, for both RELIABLE and BEST_EFFORT QoS with100 byte background tra�c packet sizes (Figures 5c & 5d)our SDN controller is apparently not able to enforce thespecified QoS. After an investigation we discovered that itwas not the network dropping packets but the subscriberhost itself: The host’s NIC and kernel could not keep up withthe packet rate induced by the small packets as confirmedby the rx_no_buffer_count and rx_missed_errors kerneldriver statistics.

Figure 6 shows the normalized latency of each update forruns of the system with a variety of di↵erent QoS settings.Figure 6a shows the “idealized” behavior: BEST_EFFORT QoSwith no interference from the tra�c generator. The nor-malized latency in the idealized case remains close to 0 forthe entire run. The normalized latency is actually usuallyslightly less than 0. How can this be? Recall that the ex-pected arrival time is computed relative to the time the firstupdate is received. The publisher starts out sending thefirst few updates with a frequency slightly greater then thespecified MIN_SPEARATION (verified by inspecting the sendingtimestamps). It is unclear why this behavior occurs but wehypothesize speed increases are due to optimizations result-ing from JIT compilation of the Java application or cachee↵ects in the CPU.

Figures 6b, 6c, 6d, 6e, and 6f all show the normalized la-tency under a variety of QoS configurations while the systemis subjected to periodic network contention and no SDN sup-port. In each case, the periodic interference pattern is visiblein the normalized latency. For BEST_EFFORT QoS (Figure 6(b)) updates are delayed or dropped while the tra�c gener-ator is active. Interestingly, the normalized latency of thereceived updates plateaus. Why is this? We believe thatthe plateau results from the size of the egress queue on theswitch: Updates that would take longer due to queue con-tention are dropped because the queue is already at capacity.

For RELIABLEQoS (Figures 6c, 6d, 6e, and 6f) we again seehow the network contention, reliable multicast, and DESTI-NATION_ORDER causes a cascade of delays due to dropped up-dates. This e↵ect is most striking when the HISTORY_DEPTH= 1 (Figure 6f): The network contention causes an updateto be missed early in each of the tra�c generator’s activeperiods which causes all subsequent messages to be delayed.However, once the dropped update is recovered subsequentupdates flow steadily.

When our SDN controller is used to manage the tra�cflows, the normalized latency for both BEST_EFFORT and RE-LIABLE runs closely matches the idealized run where thereis no background tra�c.

6. RELATED WORKThe most closely related work is by King et al. [8]. King et

al. proposed a custom publish-subscribe middleware specif-ically designed to work with OpenFlow networks to enforceend-to-end QoS. While [8] provides the results of a prelim-inary evaluation demonstrating the potential of using SDNto enforce QoS in publish-subscribe networks no comparisonwas made to an “industrial strength” middleware with ad-vanced reliability and resilience features like DDS: The cus-tom middleware used in [8] only supports unreliable UDPmulticasting to transport data from publisher to subcriber.While this limitation does not cast doubt on the evaluationpresented in [8] it does mean that King et al.’s results mightnot have applied to more realistic middleware implementa-

tions. The experimental evaluation in this paper should beconsidered confirmation, using industrial strength middle-ware, of ideas originally proposed in [8].

Also related is the work by Bertaux et al. in [2]. In [2],Bertaux et al. describes an OpenFlow SDN architectureintended to capture DDS QoS and optimize the network atLayers 2 & 3 to achieve the specified QoS. Unfortunately, [2]does not provide many details of their system nor do theyprovide any sort of experimental evaluation.

Recently there have been a number of e↵orts to augmentDDS with the ability to optimize its QoS parameters on thefly [6]. These approaches work by adding a“QoS monitor”tothe middleware. The QoS monitor reacts to changes in theactual runtime performance of the system and attempts toglobally optimize performance by tweaking the local perfor-mance of each DDS participant (e.g., causing the publishersto slow down or speed up how often they transmit updates).

[13] performed a number of di↵erent performance eval-uations of DDS middleware implementations. Unlike ourevaluation the experiments in [13] do not specifically howwell the middleware performs under faulty or overload con-ditions. Instead, the goal was to evaluated the average casebehavior of the implementation and to test its scalability(i.e., the number of DDS participants that could be e↵ec-tively managed).

7. CONCLUSION & FUTURE WORKIn this paper we evaluated how a real DDS middleware im-

plementation performs under network overload conditions.Because an increasing amount of critical infrastructure isdepending on DDS for correct function, it is important tounderstand how the middleware compensates for less thanideal conditions. As far as we know, this is the first evalua-tion of its kind for DDS middleware.

We also presented solution that uses software defined net-working to fully “lift” timing properties into the DDS pro-gramming abstraction: Our SDN controller ensures that therequested QoS is guaranteed. Our solution has a number ofadvantages, including the fact that it can work with com-modity o↵ the shelf networking hardware and does not re-quire any modification to the DDS middleware implemen-tation. Our experiments indicate that our SDN controlleris able to e↵ectively mitigate the overload conditions thatwould normally negatively impact the performance of a DDSpublish-subscribe system.

Our results are promising but preliminary. For example,our experiments only involved a network with one switchand one real-time flow. Real publish-subscribe applicationscan span many networking switching elements and involvemany thousands of publish-susbcribe flows, each with theirown QoS requirements. One very important direction forfuture work is larger scale experimental evaluations, both interms of the number of network switching elements and thenumber of flows with real-time requirements.

Lastly, while this paper focused on using SDN to enforcethe timing properties of publish-subscribe systems one canimagine that SDN could also be used to enforce other non-functional properties such as security. For example, manysecurity-sensitive systems require that classified data onlyflow between classified nodes. SDN could automatically en-force dataflow properties in the network itself by only gener-ating flow rules that satisfy the security requirements. Inves-tigating how publish-subscribe middleware and SDN could

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

Del

ay(s

)....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

(a)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

...............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

(a) (b)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

............................................................................................................................................................................................................................................................................................................................................................................................................................................................................. ......................................................................... .......................................................................... ............................................................................ ...............................................................................

(a) (b) (c)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

Del

ay(s

)

..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.......................................................................................................................

........................................................................................................................

.........................................................................................................................

............................................................................................................................

(a) (b) (c)

(d)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.........................................................................................................................................................................

..........................................................................................................................................................................

............................................................................................................................................................................

................................................................................................................................................................................

(a) (b) (c)

(d) (e)

0.01.02.03.04.05.06.0

0 1000 2000 3000 4000Sequence

...........................................................................................................................................................................................................................................................................................................................................................................................................................................................................

...................................................................................................................................................................................................................................................................................................................................................................................................................

....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

.............................

(a) (b) (c)

(d) (e) (f)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000

Del

ay(s

)

Sequence

....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

(a) (b) (c)

(d) (e) (f)

(g)

-0.10.00.10.20.30.40.50.6

0 1000 2000 3000 4000Sequence

....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

(a) (b) (c)

(d) (e) (f)

(g) (h)(a) Best-effort(No-load) w/o SDN(b) Best-Effort w/o SDN(c) Reliable(depth=1) w/o SDN(d) Reliable(depth=50) w/o SDN(e) Reliable(depth=100) w/o SDN(f) Reliable(depth=∞) w/o SDN(g) Best-Effort w/ SDN(h) Reliable(depth=1) w/ SDN

(a) (b) (c)

(d) (e) (f)

(g) (h)

Figure 6: Normalized update latency vs. sequence number. 1400 byte background tra�c packets.

be combined to create a robust security architecture couldbe interesting future work.

8. REFERENCES[1] L. Atzori, A. Iera, and G. Morabito. The internet of

things: A survey. Computer networks,54(15):2787–2805, 2010.

[2] L. Bertaux, A. Hakiri, S. Medjiah, P. Berthou, andS. Abdellatif. A dds/sdn based communication systemfor e�cient support of dynamic distributed real-timeapplications. In Distributed Simulation and Real TimeApplications (DS-RT), 2014 IEEE/ACM 18thInternational Symposium on, pages 77–84. IEEE, 2014.

[3] DoD. The Data Distribution Service: Reducing CostsThrough Agile Integration. Technical report, U.S.Department of Defense, 2011.

[4] P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M.Kermarrec. The many faces of publish/subscribe.ACM Computing Surveys (CSUR), 35(2):114–131,2003.

[5] M. Gill. Rti data distribution service selected forupgrade of grand coulee dam‘s control system.Rueters, 2009.

[6] J. Ho↵ert, A. Gokhale, and D. C. Schmidt. Timelyautonomic adaptation of publish/subscribemiddleware in dynamic environments. Innovations andApproaches for Resilient and Adaptive Systems, page172, 2012.

[7] R.-T. Innovations. Rti connext dds professional, 2014.[8] A. L. King, S. Chen, and I. Lee. The middleware

assurance substrate: Enabling strong real-timeguarantees in open systems with openflow. InObject/Component/Service-Oriented Real-TimeDistributed Computing (ISORC), 2014 IEEE 17thInternational Symposium on, pages 133–140. IEEE,2014.

[9] J.-Y. Le Boudec and P. Thiran. Network calculus: atheory of deterministic queuing systems for theinternet, volume 2050. Springer Science & BusinessMedia, 2001.

[10] N. McKeown, T. Anderson, H. Balakrishnan,G. Parulkar, L. Peterson, J. Rexford, S. Shenker, andJ. Turner. Openflow: enabling innovation in campusnetworks. ACM SIGCOMM ComputerCommunication Review, 38(2):69–74, 2008.

[11] G. Pardo-Castellote. Omg data-distribution service:Architectural overview. In Distributed ComputingSystems Workshops, 2003. Proceedings. 23rdInternational Conference on, pages 200–206. IEEE,2003.

[12] J. Plourde, D. Arney, and J. M. Goldman. Openice:An open, interoperable platform for medicalcyber-physical systems. In Cyber-Physical Systems(ICCPS), 2014 ACM/IEEE International Conferenceon, pages 221–221. IEEE, 2014.

[13] S. Sierla, J. Peltola, and K. Koskinen. Evaluation of areal-time distribution service.

[14] J. A. Stankovic. Research directions for the internet ofthings. Internet of Things Journal, IEEE, 1(1):3–9,2014.

[15] S. Vinoski. Corba: integrating diverse applicationswithin distributed heterogeneous environments.Communications Magazine, IEEE, 35(2):46–55, 1997.

[16] A. Vukojevic, S. Laval, and J. Handley. An integratedutility microgrid test site ecosystem optimized by anopen interoperable distributed intelligence platform.In Innovative Smart Grid Technologies Conference(ISGT), 2015 IEEE Power & Energy Society, pages1–5. IEEE, 2015.