Design and development of a proof-of-concept platooning application using theHIDENETS architecture

Luıs Marqueslmarques@lasige.di.fc.ul.pt

FC/UL∗

Antonio Casimirocasim@di.fc.ul.pt

FC/UL

Mario Calhamjc@di.fc.ul.pt

FC/UL

Abstract

This paper describes the design and development of aproof-of-concept platooning application, which operates ina mobile and dynamic environment and makes use of ar-chitectural and middleware solutions that were proposed inthe scope of the HIDENETS project. With this application itis possible to demonstrate the practical feasibility of a hy-brid system architecture, with realms of operation with dis-tinct synchrony properties, and the benefits of adopting sucharchitecture. In particular, we show that it is possible to im-prove the performance and behavior of the platooning ap-plication, which operates over an intrinsically uncertain en-vironment (due to mobility and wireless communication),and still secure fundamental safety-critical requirements.

1. Introduction

The continuous advances in embedded, sensor and wire-less technologies are key to the development of distributedapplications and mobility-aware services in ubiquitouscommunication scenarios, such as car-to-car communica-tion scenarios.

On the other hand, the typical characteristics of thesescenarios, involving mobile nodes that communicatethrough wireless networks, possibly using general pur-pose operating systems and COTS-based devices, leadto uncertainty on the temporal behavior and on the reli-ability of the operations taking place therein. Therefore,in these scenarios it is not easy to provide highly re-silient and available services and applications.

The well defined objective of HIDENETS [5], the IST-FP6 project in which our work has been developed, is pre-cisely to address this problem, in particular through the def-inition of appropriate architectural constructs and resilienceservices.

In this paper we focus on the platooning application,which was one of the proof-of-concept applications that

∗ Faculdade de Ciencias da Universidade de Lisboa. Bloco C6,Campo Grande, 1749-016 Lisboa, Portugal. Navigators Home Page:http://www.navigators.di.fc.ul.pt. This work was partially supportedby the EC, through project IST-FP6-STREP-26979 (HIDENETS) andby the FCT through the Multiannual Funding and the CMU-PortugalPrograms.

were developed to demonstrate the project results. Briefly,the objective of platooning is to ensure that a set of well-known cars follow each other as close as possible in a waythat improves traffic flow, road capacity and fuel consump-tion [7]. In contrast with typical control approaches, whichonly make use of locally produced information (from inter-nal car sensors), we consider a platooning application thatuses vehicle-to-vehicle communication to obtain additionalinformation and thus be able to make more “intelligent” de-cisions and achieve an optimized behavior with respect tothe real surrounding context. Interestingly, and very impor-tantly, this optimized behavior is achieved without compro-mising any safety-critical property.

Through the proof-of-concept platooning application de-scribed in this paper, we demonstrate the feasibility of somesolutions developed in the scope of HIDENETS. In partic-ular, we demonstrate:

• the feasibility of a hybrid system architecture in whichdifferent parts of the system have different synchronyproperties;

• the benefits of using such hybrid architecture and somesupporting services in the construction of a representa-tive platooning application, showing how to achieve animplementation that combines efficiency with safety.

The paper is organized as follows. In Section 2 webriefly present the HIDENETS project and introduce theHIDENETS hybrid architecture. Then, Section 3 providesa general description of the platooning scenario and Sec-tion 4 presents the application design and implementation.The demonstration results are addressed in Section 5 andwe make final remarks in Section 6.

2. HIDENETS Architecture

HIDENETS is a research project, funded by the Euro-pean Union, which aims to develop and analyze end-to-endresilience solutions for distributed and mobile applicationsin ubiquitous communication scenarios, assuming highlydynamic, unreliable communication infrastructures [5]. Theproject addresses fault tolerance mechanisms at the middle-ware and communication layers, as well as methodologiesto support their evaluation and testing [3].

In HIDENETS we followed a hybrid distributed systemmodel approach, with different parts of the system beingdescribed by different synchrony and fault models. This hy-bridization was inspired in the Wormholes model [8], whichassumes that it is possible to construct a part of the systemwith improved properties (the wormhole), which is capa-ble of performing some actions in a faster, more predictableor more secure way than it is possible outside of it. Fig-ure 1 provides a simplified view of the HIDENETS archi-tecture.

HW

Applications

Network Interface Cards

(COTS)

Optimized Network Protocols

Complex Resilience Middleware Services

Trusted

Resilience

Services

Resilience HW &

Wormhole Comm.

Interfaces

Video

StreamingPlatooning Whiteboard

Reconfigu-

ration Mgr.Replication Mgr.

Diagnostics Mgr.

Cooperative Data

Backup

Timely Timing

Failure Detection

Self-aware

Clock

SAF interfaces Hidenets Interfaces

…

…

…

Multi-channel/-radio

Management

Gateway selection &

dependable Routing

Topology Control &

Neighbor discovery

Reliable Broadcast

…

Figure 1. HIDENETS architecture.

There are two distinct domains in this architec-ture: the bulk or “payload” part of the system, implementedin a general purpose computing node, includes the ap-plications, complex middleware functions and optimizednetwork protocols over standard communication infras-tructures; the “wormhole” part of the system consists ina set of simple and resilient services that must be imple-mented separately, typically within a subsystem with itsown computational resources, possibly dedicated commu-nication channels and a clearly defined interface to thepayload part. In this paper, and in the context of the pla-tooning application, we refer to a particular implementa-tion of a wormhole and we briefly describe the servicesthat it provides to the platooning application software run-ning in the payload.

We must note that this architecture is well suited to realvehicles. Modern cars are equipped with a huge amount ofsensors, connected to central control units through one orseveral networks, and grouped in accordance to their pur-pose and/or criticality. A car can be seen as a heterogeneouscomputational environment, with distinct parts, each withtheir own synchrony and reliability properties and with theirown purpose. In such real car, the wormhole part of our sys-tem would have to be integrated as an extension of the crit-ical and predictable car control subsystems, from where itcould reach the relevant actuators and perform timely op-

eration. On the other hand, the payload part would be thegeneral purpose computational infrastructure, where non-critical applications (e.g., infotainment, entertainment, pas-sive security) are executed and provided to passengers.

3. Platooning scenario and assumptions

We consider a platooning application for a predefined setof cars. Each car is equipped with a front proximity sensor,communicates over a wireless network and obtains its po-sition through a GPS receiver. For simplicity we only con-sider positioning on a single dimensional axis and we as-sume that lateral steering is not required, or is automaticallygranted.

In the specification of the platooning application we de-fine two fundamental requirements. A safety-critical re-quirement is that cars have to maintain a minimum distanceto the front car, defined as ∆min. This distance will varyover time, depending on the car speed and environment con-ditions. A non safety-critical requirement is that each carhas to maintain a maximum distance to the rear car, definedas ∆max, to avoid platoon fragmentation. The ∆max re-quirement depends on the maximum communication rangeassumed for the wireless network.

We further consider that the driver has control of whenthe platooning application becomes active, by pushing aplatoon activation button (similarly to a traditional cruisecontrol system).

4. Design and Implementation

4.1. Overview of platooning scenario

In the conceptual platooning scenario we have a set ofmobile entities that are at the same time computational ele-ments (distributed nodes that communicate with each other)and physical entities (cars) that obey physical laws of move-ment, are operated by a human (driver) and interact with thephysical environment through sensors (speed, distance, po-sition, etc) and actuators (accelerator and brakes). All thesecomponents are represented in Figure 2, which we now ex-plain.

The bottom part of the figure represents the physical en-vironment, emulated using The Open Racing Car Simula-tor (TORCS) [6], which offers a graphical interface to ob-serve representations of the cars, their positions and move-ments (see Section 4.2).

On top of that we have the computational representa-tion of the cars where five cars are illustrated. Each car (aHIDENETS node) is composed by an embedded real-timedevice (the wormhole) and a generic laptop PC or PDA (thepayload), which are connected through a serial RS232 in-terface. This hybrid node executes all the software that con-trols the car behavior (the platooning application) and pro-vides interfaces for the driver (accelerator, brake and pla-toon activation). Sensing and actuation interactions take

Ad hoc environment

PositionTime

Speed

Speed controlBrake

PlatooningApplication(car view)

(Ethernet)

Emulator network

Environment Emulator(global view)

HIDENETSnode

802.11 network

RS

23

2

RS

23

2

RS

23

2

RS

23

2

RS

23

2

Topology Emulator

Figure 2. Implementation overview.

place between each node (car) and the environment emu-lator. Actuation commands include acceleration and brakecontrol, while sensing information includes position, timeand speed.

Finally, the demonstration setup also includes the wire-less communication network through which the cars ex-change information (top part of Figure 2). For the purposesof the demonstration a standard 802.11 network could beused. However, since the nodes in the demonstration arestatic and physically close to each other, the communica-tion properties would not depend on the distance betweencars, maintained by the environment emulator. Therefore, inthe demonstration we use a topology emulator (also devel-oped in HIDENETS), which models a 802.11 network andis connected to TORCS to receive information from the po-sitioning of communication entities in run-time.

4.2. Environment emulator

For the simulation of the platooning environment andphysics we adapted an open source 3D racing simula-tor, named The Open Racing Car Simulator (TORCS) [6].TORCS simulates a race where a configurable number ofhuman and computer drivers each control a chosen car. Inevery iteration, the simulator provides information abouteach car situation and evaluates available commands for ac-celeration and braking.

Several changes were made to better fit the simulatorto the platooning scenario. For instance, no steering com-

mands are considered since all cars steer to follow a com-mon path on the right side of the road (controlled by the em-ulator on its own).

4.3. Platooning application architecture

The platooning application itself is split in two parts: amajor part residing on the payload subsystem (on the lap-top PC or PDA) and a simpler part in the wormhole subsys-tem (the real-time embedded device that represents the pre-dictable part of the car infrastructure). This separation canbe seen in Figure 3.

� � �

� � � � � � � � � � � � � �

� � � � � �� � � � � � � �

� � � � � � � � � �

� � � � � � � � �� � � � � �

� � � � � � � � � � �

� � � � � � � � �

� � � � �

� � � � � � � � �

� � � � � � � � � � � � �

� � � � � � � � � � � � � � � � � � � �

! ! " �� � � � � � # �

� � � � � � � � � �

� � $ � � �� # � � �

% & ' &( &

� � �� � � � � �� � � � � �

� � � $ � � � � � � � � � � � � � � �

) � � � � � � �! � � � � �

� � � � � � �

* � �� � $ � � � �� � +

� �

�� ��������

, ) � - . - ! � � � � � $ � � �

� � � � � � � � � � � � � �

Figure 3. Application architecture.

According to the previously described HIDENETS ar-chitecture, the payload is where generic applications exe-cute and where arbitrarily complex computations can takeplace. Therefore, the platooning application can be made assophisticated as needed in this part of the system, by execut-ing complex tasks for decision making and for communica-tion, dealing with potentially large amounts and variety ofdata. The trade-off is the lack of timeliness guarantees, typ-ical in asynchronous systems where an uncertain numberof applications compete for the available resources (CPU,memory and network).

Because of that, the platooning application is comple-mented by baseline mechanisms, implemented within thewormhole subsystem, which assure a safe and timely re-sponse to relevant events, namely timing failures causedby the uncertain temporal behavior of the payload part, aswe explain in Section 4.4.3. The wormhole implements apayload-wormhole information gateway, though which sen-sor information is provided to the payload and platooningspeed control commands are received.

4.4. Wormhole subsystem

The wormhole was implemented using a dedicated real-time device and operating system. The hardware employedwas Lantronix’s UDS100 device, which is driven by a48MHz 8086 compatible CPU and has 256Kb of RAM.The device provides Ethernet and serial connections, whichwere used to communicate with the environment emula-tor and the payload, respectively. Additionally, two pro-grammable LEDs, a green and a red one, were used to showthe status of the Timely Timing Failure Detection service.

The operating system of the device is proprietary and notfully disclosed, but it is documented as an O.S. with real-time capabilities, which we have explored in the wormholeimplementation.

The wormhole essentially includes a task to process in-coming requests arriving at the payload-wormhole inter-face (where admission control is performed), a task to pro-cess the periodic interactions with the environment emula-tor (read sensors and write to actuators) and a task to pro-cess internal events, such as timing failures detected by theTimely Timing Failure Detection (TTFD) service. The max-imum execution time of these tasks can be bounded, render-ing the wormhole a synchronous component, as assumed inthe HIDENETS model.

In the wormhole implementation we included three ba-sic services: an Authentication service, a Reliable and Self-Aware Clock service and a TTFD service (which uses Du-ration Measurement facilities). These are presented in thefollowing sections.

4.4.1. Authentication In real platooning scenarios it maybe necessary to deal with malicious behaviors and with falsedata originating from non-trusted sources. Although thiswas not the main concern of the HIDENETS project, we ex-ploited the availability of a wormhole subsystem with betterproperties to add an Authentication service.

In essence, the idea is to use a scheme based on a public-key infrastructure model, in which the wormhole keeps aprivate key that is used to sign sensible information (e.g.,position, speed) collected from sensors before this informa-tion is sent to the payload subsystem. Given that, payloadapplications, and the platooning in particular, may authen-ticate received information before using it. We note, how-ever, that we are not concerned with security issues in thisdemonstration.

4.4.2. Reliable and Self-Aware Clock The Reliable andSelf-Aware Clock service provides both current time andcurrent synchronization uncertainty, i.e. an estimation of thedistance of the local clock from an external global time ref-erence [1]. This allows a client to query not only the (mostlikely) current time, but also to obtain assured minimum andmaximum time values.

In our implementation the R&SA clock is synchronizedperiodically through the GPS receiver (emulated in TORCSand read as another internal car sensor), thus using a globaltime reference. In the platooning application the R&SA

Clock service is used to timestamp the positioning datashared with other cars. This allows the cars to relate allthe received positioning information in a coherent timeline,while also using the most conservative timestamp values,for maximum safety.

4.4.3. Timely Timing Failure Detection In order to en-sure the safety properties of platooning, it is essential thatactuation commands that determine the speed of the car areexecuted in a timely way. However, it is clear that the com-mands issued by the platooning decision control task in thepayload subsystem may be subject to unpredictable delaysbefore being executed by the wormhole subsystem.

Because of this, the TTFD service is used to observetimed executions and react upon the occurrence of timingfailures. It works like a switch that determines if the car canbe controlled by the payload decision control task or, in-stead, if the safety distance control mechanism becomes ac-tive (see Figure 3).

While the car is under the drivers’ control, only the safetydistance control mechanism is active. Then, when platoon-ing is activated, the payload control task sets up the TTFDservice, specifying a deadline for the next control commandthat will be sent to the wormhole. If the command indeed ar-rives before the specified deadline, then it is safe to send thiscommand to the actuators (which is done, for predictabil-ity reasons, at the deadline instant). While the payload istimely, the information gateway is allowed to accept com-mands received from the payload and the safety distancecontrol mechanism is disabled. At the same time, the TTFDis re-armed for a new deadline (specified by the payload,along with the actuation command). This process is con-tinuously repeated until a timing failure occurs. The tim-ing failure is timely detected by the TTFD, which imme-diately activates the safety distance control mechanism andprevents any further commands from the payload to be ac-cepted.

Note that the Duration Measurement service is simplyused to calculate the time elapsed since the TTFD was re-quested to observe the timeliness of a timed action, and theinstant in which the action terminates.

4.5. Payload subsystem

The payload is structured as two main tasks, which sharean internal data structure with platooning-related informa-tion. One task is concerned with exchanging informationwith the remaining cars (each message contains an iden-tifer of the car and time stamped information about posi-tion and speed), storing this information in the shared datastructure. This task is always running and disseminating thislocal information. The other task runs a periodic decisioncontrol algorithm, which implements the optimized behav-iors of the platooning using the information contained in theshared data structure and interacting with the Intrusion Tol-erant Agreement and the QoS Coverage services. These willbe explained next.

4.5.1. Decision control algorithm The decision controltask wakes up periodically, with a period that is made shortenough to allow a smooth control of the vehicle, but suffi-ciently large to accommodate the reception of new informa-tion from other vehicles in between. In each iteration, thetask starts by requesting to the wormhole updated informa-tion about local time, position and speed, which is stored inthe shared information structure. Then, it uses all the avail-able information concerning vehicle positions and speeds,in order to determine a new speed command to be sent tothe wormhole. The algorithm makes worst case assump-tions about the behavior of neighbor vehicles (in particu-lar, assuming that they have braked to stop just after send-ing their last positioning/speed information). It also consid-ers the deadline for sending the current actuation command(this deadline has been previously set on the TTFD service),and the deadline for sending the next actuation command(when there will be a new opportunity for setting a newspeed, or when will the car will switch into safe mode af-ter receiving the current command). The latter will be sentalong with control command to re-arm the TTFD service.

The communication task is simpler. It is activated when-ever some message is received from another node, and pe-riodically disseminates local information that it reads fromthe shared data repository.

4.5.2. Intrusion tolerant agreement The Intrusion Tol-erant Agreement service is used, in essence, to allow pla-toon members to agree on a common platoon speed. It is in-voked whenever a driver activates the platooning mode. Inthe implementation we used an adapted version of the ran-domized, intrusion-tolerant, asynchronous vector consen-sus, from the RITAS protocol stack [4]. Although this ser-vice is meant to deal with malicious participants, in this spe-cific demonstration we have other goals and therefore donot consider these capabilities, just the possibility of reach-ing consensus on a value.

4.5.3. QoS coverage As mentioned, the nodes of theplatoon exchange messages through the wireless net-work. These messages are subject to unpredictable delays,and may even be lost. On the other hand, the “fresh-ness” of these messages is important in the decision controlalgorithm: the optimality of the control decisions de-pends on this freshness. Since the decision control task ex-ecutes periodically, and in each execution it sets thenext control set-point (deadline for actuation), this dead-line should be calculated using a prediction on whenthere will be new information available from the remain-ing cars.

The QoS Coverage service is helpful as it provides hintson the message delivery delay that should be considered,holding with a certain specified probability. Using this ser-vice, any changes in the communication environment af-fecting communication delays will be reflected in the con-sidered value and hence in the algorithm deadlines. A de-tailed description of the probabilistic framework used in thisservice can be found in [2].

5. Demonstration

In this section we describe and analyze the results ob-tained by running our proof-of-concept platooning applica-tion in a test-bed with 3 nodes, plus the environment emu-lator node, as shown in Figure 4.

Figure 4. The complete demo setup.

5.1. Normal platooning behavior

When the demonstration is initiated, cars are controlledby their drivers. In this state the safety distance controlmechanisms is active, but driver commands always takeprecedence. In the demo we show that a driver has free con-trol of the car and is able to crash into the front car. With-out driver intervention, and given the safety distance con-trol, the car will autonomously brake. We also show that inthis state there is no active mechanism in effect to preventplatoon fragmentation.

Once the drivers activate the platooning, the cars agreeon a speed value and the platoon is formed. We show thatcars can follow each other very closely, even at high speeds,and when the car in the middle of the platoon brakes at fullintensity the follower will also stop, without colliding. Onthe other hand, the decision control algorithm in precedingcar will eventually stop this cars also, to secure the maxi-mum distance property.

5.2. Dealing with timing failures

To observe the behavior of the TTFD service, and thedetection of timing failures, we use the green and red LEDlights of the embedded device. When the TTFD service isactivated and does not detect any timing failure, the greenLED is blinking. But when a timing failure is detected, thered LED is turned ON. Therefore, it is possible to know ifa car is being controlled by the platooning application or bythe safety control algorithm, just by observing the LEDs.

time.When a heavyweight task is ran on the payload system it

will compete with the platooning application for CPU time,

and may cause timing failures to occur. In the demonstra-tion we reproduce this behavior and we show that a tim-ing failure is promptly detected. The red LED turns on andthe safety distance control mechanism is activated, forcingthe car to brake in order to preserve the baseline, non opti-mal, safety distance.

5.3. Dealing with communication uncertainty

To illustrate the role of the QoS Coverage service, wefirst show what happens when we artificially inject some de-lays in the ad-hoc network, while not using the QoS Cover-age service. In this situation the decision control algorithmsometimes issues braking commands, even when not neces-sary (i.e., even when the front car is moving with a constantspeed). This happens because control decisions are takenusing “old” information about the front car, since updatedinformation is delayed.

When the QoS Coverage service is activated, we observethat when delays are injected the decision control algorithmsends the necessary commands to force a greater distance tothe front car. This extra distance allows to extend the controlperiod and the time during which cars can wait for incom-ing “fresh” information, reducing the probability that com-munication delays will force unnecessary, inconvenient andconstant braking manoeuvres.

5.4. Dealing with temporal inconsistency

In the event of a network crash the information aboutneighbor cars will not be updated. In practice, as we showin the demonstration, the effect is that cars will eventuallystop, either because they believe (from the old available in-formation) that they have a car in front of them, or waitingfor a car that they believe has stopped way behind.

Note that in the case of this platooning application, tem-porally inconsistent information does not affect any safety-critical property – it only compromises the liveness of thesystem. In fact, this is true because we assume that carsalways move forward, but never backwards. In other sce-narios, using temporally inconsistent information may havevery harmful effects.

Interestingly, the hybrid system architecture that weuse to implement the platooning application also facil-itates dealing with temporal inconsistency. In fact, theproblem mentioned above occurs because the decision con-trol task is running in a timely way, and therefore is allowedto control the car, despite the outdated view of the sur-rounding environment. The solution consists in simplystopping the decision control task, automatically transfer-ring the control to the safety distance control mechanism.For doing that, the decision control task evaluates the fresh-ness of the available information in order to decide when tosilently stop.

In the demonstration we show the behavior of the pla-toon in the two situations, with and without this tempo-ral inconsistency detection mechanism activated. Therefore,

when we crash the network preventing any further commu-nication, we observe that with the temporal inconsistencydetection the cars keep on moving, just ensuring a (non-optimistic) safe distance to the front car, until they eventu-ally stop if the driver does not take get in control.

6. Conclusions

In this paper we analyzed the design and implementa-tion of a distributed platooning application, built using theHIDENETS architecture. We described the several compo-nents that are involved in the proof-of-concept prototypedemonstration and showed how the use of the HIDENETShybrid architecture and its supporting services enabled usto solve the difficult problem of constructing an arbitrarilycomplex, temporally uncertain control application while se-curing fundamental safety-critical requirements.

We presented a hybrid system design using an inex-pensive, real-time embedded wormhole, which neverthelessprovides the necessary characteristics that allow our objec-tives to be fulfilled.

Finally, we described the steps of a demonstration thatshows the correct operation of the application in realisticscenarios and highlights the improvements that may effec-tively be achieved with our approach.

Acknowledgments

We wish to warmly acknowledge the comments received fromour colleagues in the HIDENETS project, in earlier stages of thiswork. Special acknowledgments must be given to Lorenzo Falaiand Andrea Bondavalli, who proposed and defined the R&SAClock service, to Jose Rufino, for his work in the definition ofthe TTFD service, to Monica Dixit, for her work in the QoS Cov-erage service and to Henrique Moniz, for his work in the IntrusionTolerant Agreement service.

References[1] A. Bondavalli, A. Ceccarelli, and L. Falai. Assuring resilient time syn-

chronization. In Proc. of the 27th International Symposium on Reli-able Distributed Systems, pages 3–12, Napoli, Italy, oct 2008.

[2] A. Casimiro, P. Lollini, M. Dixit, A. Bondavalli, and P. Verıssimo.A framework for dependable qos adaptation in probabilistic environ-ments. In 23rd ACM Symposium on Applied Computing, Dependableand Adaptive Distributed Systems Track, pages 2192–2196, Fortaleza,Ceara, Brazil, Mar. 2008.

[3] J. A. et al. Revised reference model. June 2007. EU FP6 IST projectHIDENETS, Deliverable D1.2.

[4] H. Moniz, N. F. Neves, M. Correia, and P. Verıssimo. Randomizedintrusion-tolerant asynchronous services. In DSN ’06: Proceedings ofthe International Conference on Dependable Systems and Networks(DSN’06), pages 568–577, jun 2006.

[5] H. W. site. http://www.hidenets.aau.dk/.[6] T. W. site. http://torcs.sourceforge.net/.[7] P. Varaiya. Smart cars on smart roads: Problems of control. IEEE

Transactions on Automatic Control, 38(2):195–207, 1993.[8] P. Verissimo. Travelling through wormholes: a new look at distributed

systems models. SIGACT News, 37(1):66–81, 2006.