Post on 15-Jan-2015
description
OpenSplice | DDS Tutorial-- Part I --
Angelo CORSARO, Ph.D.Chief Technology Officer OMG DDS Sig Co-Chair
PrismTechangelo.corsaro@prismtech.com
Grasping the Idea
DDS is a standard technology for ubiquitous, interoperable, secure, platform independent, and
real-time data sharing across network connected devices
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Data Distribution Service (DDS)
• DDS provides a Global Data Space abstraction that allow applications to autonomously, anonymously, securely and efficiently share data
• DDS’ Global Data Space is fully distributed, highly efficient and scalable
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Data Distribution Service (DDS)
• DataWriters and DataReaders are automatically and dynamically matched by the DDS Discovery
• A rich set of QoS allows to control existential, temporal, and spatial properties of data
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Fully Distributed Data Space
Conceptual Model Actual Implementation
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
TopicDQoS
TopicDQoS
TopicAQoS
Fully Distributed Data SpaceThe communication between the DataWriter and the DataReader can use UDP/IP (Unicast and Multicast)or TCP/IP
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Key Highlights
• Elegant and High Level Data Sharing Abstraction
• Polyglot and platform independent
• Java, Scala, C, C++, C#, JavaScript, CoffeeScript etc.
• Android, Windows, Linux, VxWorks, etc.
• Peer-to-Peer by nature, Brokered when useful
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Key Highlights
• Content and Temporal Filtering (both sender and receiver filtering supported)
• Queries
• 20+ QoS to control existential, temporal, and spatial properties of data
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Key Highlights
• High Performance and Scalable
• ~35-50 usec latency*
• 7M msgs/sec node-to-node throughput*
(*) Performance measured on Linux Host with i7 processor on a 1Gbps Ethernet network
Who is Using DDS?
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Use Cases
Integrated Modular Vetronics Training & Simulation Systems Naval Combat Systems
Air Traffic Control & Management Unmanned Air Vehicles Aerospace Applications
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Use Cases
Agricultural Vehicle Systems
Train Control Systems Complex Medical Devices
Smart CitiesLarge Scale SCADA Systems
High Frequency Auto-Trading
Decomposing DDS
Information Organization
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Domain
• DDS data lives within a domain
• A domain is identified with a non negative integer, such as 1, 3, 31
• The number 0 identifies the default domain
• A domain represent an impassable communication plane
DDS Domain
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Partitions
• Partitions are the mechanism provided by DDS to organize information within a domain
• Access to partitions is controlled through QoS Policies
• Partitions are defined as strings:
• “system:telemetry”
• “system:log”
• “data:row-‐2:col-‐3”
• Partitions addressed by name or regular expressions:
• ”system:telemetry”
• “data:row-‐2:col-‐*”
Partitions
Information Definition
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Topic• A Topic defines a domain-wide information’s class
• A Topic is defined by means of a (name, type, qos) tuple, where
• name: identifies the topic within the domain
• type: is the programming language type associated with the topic. Types are extensible and evolvable
• qos: is a collection of policies that express the non-functional properties of this topic, e.g. reliability, persistence, etc.
TopicType
Name
QoS
...
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
...
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Topic and Instances
• As explained in the previous slide a topic defines a class/type of information
• Topics can be defined as Singleton or can have multiple Instances
• Topic Instances are identified by means of the topic key
• A Topic Key is identified by a tuple of attributes -- like in databases
• Remarks:
• A Singleton topic has a single domain-wide instance
• A “regular” Topic can have as many instances as the number of different key values, e.g., if the key is an 8-bit character then the topic can have 256 different instances
Example
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floor
• Assume we are building an active floor
• This active floor is made by a matrix of pressure sensors used to detects position, and indirectly movement
• This information is then used by the application that uses the active floor for positioning or entertainment
Cell: (i,j)
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floor
• The generic active cell can be modeled with a topic that has an instance for each value of (i,j). The topic type can be defined as:
• Each cell is now distinguishable and associated with a topic instance
Cell: (i,j)
struct TCell { short row; short column; float pressure; // in kPa};#pragma keylist TCell row column
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floor
• How can we know when something is on the cell?
• The detection can be based on the difference between the atmospheric pressure, say P0, and the pressure sensed by the cell
• We can model this as a Singleton Topic ReferencePressure defined by the type:
Cell: (i,j)
struct TReferencePressure { float pressure; // in kPa float precision; };#pragma keylist TReferencePressure
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floor• Each sensor has associated a topic
instance identified by the (row,column) coordinate -- the instance key
• Each instance produces a stream of pressure values that in DDS terms are called samples
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
struct TCell { short row; short column; float pressure; // in kPa};#pragma keylist Cell row column
Going 3D
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floors on a Building
• Let’s assume now that we have a building that uses active floors to detect presence and movement
• How can we organize the out data model?
01
23
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floors on a Building
• The first thing to do is to introduce the third dimension to our cell:
struct TCell { short row; short column; short floor; float pressure; // in kPa};#pragma keylist TCell row column floor
01
23
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floors on a Building
• As we move from a single floor to a building we need to add some more structure to our data
• We can now use:
• A Domain for each Building
• A Partition for each Floor
• A Partition for reference value, i.e. Partition
• A Partition for the configuration information, e.g. how many floors, how many rows/cols per floor
01
23
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Active Floors on a Building
• Thus the resulting structure is:
• Floor Partitions:
• “building:f-‐1”
• “building:f-‐2”
• ...
• Reference Values Partition:• “building:refvals”
• Configuration Partition:• “building:config”
01
23
0
1
2
3
4
Pressure
time
Pressure
time
Pressure
time
Producing Information
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataWriter
• A DataWriter (DW) is a strongly typed entity used to produce samples for one or more instances of a Topic, with a given QoS
• Conceptually, the DataWriter QoS should be the same as the Topic QoS or more stringent
• However, DDS does enforce a specific relationship between the Topic and DataWriter QoS
DWTypeTopic
QoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataWriter
• The DataWriter controls the life-cycle of Topic Instances and allows to:
• Define a new topic instance
• Write samples for a topic instance
• Dispose the topic instance
DWTypeTopic
QoS
Consuming Information
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• A DataReader (DR) is a strongly typed entity used to access and/or consume samples for a Topic, with a given QoS
• Conceptually, the DataReader QoS should be the same as the Topic QoS or less stringent
• However, DDS does enforce a specific relationship between the Topic and DataReader QoS
DRTypeTopic
QoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• Depending on its QoS a DataReader may provide access to:
• last sample
• last n samples
• all samples produced since the DataReader was created
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
n=3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader• Samples are stored in the DataReader
Cache
• Samples can be read or taken from the cache
• Samples taken are evicted from the cache
• Samples read remain in the cache and are simply market as read
• The cache content can be selected based on content or state. More on this later...
0 1 2 3
0
1
2
3
4
Pressure time
Pressure time
Pressure time
Putting all Together
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DDS Entities
• DomainParticipant: Provides access to a data cloud -- called a domain in DDS
• Topic: Domain-wide definition of a kind of Information
• Publisher/Subscriber: Provide scope to data sharing through the concept of partitions
• DataReader/DataWriter: Allow to read/write data for a given topic in the partitions their Subscriber/Publisher are associated with.
Domain (e.g. Domain 123)
Domain Participant
Topic
Publisher
DataWrter
Subscriber
DataReader
Partition (e.g. “Telemetry”, “Shapes”, )
Topic Instances/Samples
TaTb
Tc
Tx
Ty
T1
T1 T3
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DomainParticipant
• The DomainParticipant is the programming entity that gives access to a DDS domain
• A DomainParticipant is created as follows:
// ISO C++ DDS APIint domain_id = 18;auto dp = DomainParticipant(domain_id);
// Java 5 DDS APIint domain_id = 18;DomainParticipantFactory dpf = DomainParticipantFactory.getInstance(env)
DomainParticipant dp = dpf.createParticipant(domainId);
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Topic
• Given a DomainParticipant we can define (or discover) Topics within a domain. This can be done as follows:
• As this declaration does not explicitly provide QoS for the Topic, the default QoS will be used
// ISO C++ DDS APIauto topic = Topic<TCell>(dp, “Cell”);
// Java 5 DDS APITopic<TCell> topic = dp.createTopic(“Cell”, TCell.class);
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Publisher/Subscriber
• Publisher/Subscriber, through the Partitions they are associated with, define the scope of a write/read operation
• Partitions association is done through the Partition QoS Policy
• This association can be defined as a list of string as well as a list of regular expressions
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Publisher/Subscriber
• Definition of a Publisher/Subscriber in the default partition:
• Definition of a Publisher/Subscriber with Partition settings
// ISO C++ DDS APIauto pub = Publisher(dp);
// ISO C++ DDS APIauto pub_qos = dp.default_publisher_qos() << Partition(”af:telemetry”);auto pub = Publisher(dp, pub_qos);
// ISO C++ DDS APIauto sub = Subscriber(dp);
// ISO C++ DDS APIauto sub_qos = dp.default_subscriber_qos() << Partition(”af:telemetry”);auto sub = Subscriber(dp, sub_qos);
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Publisher/Subscriber
• Definition of a Publisher/Subscriber in the default partition:
• Definition of a Publisher/Subscriber with Partition settings
// Java 5 DDS APIPublisher pub = dp.createPublisher();
// Java 5 DDS APIPublisherQos pubQoS = dp.getDefaultPublisherQos() .with(pf.Partition(”af:telemetry”));
Publisher pub = dp.createPublisher(pubQoS);
// Java 5 DDS APISubscriber sub = dp.createSubscriber();
// Java 5 DDS APISubscriberQos subQoS = dp.getDefaultSubscriberQos() .with(pf.Partition(”af:telemetry”));
Subscriber sub = dp.createSubscriber(subQoS);
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataWriter
• A DataWriter with default QoS can be declared as follows:
// ISO C++ DDS APIauto dw = DataWriter<TCell>(pub, topic);// Write the cell c(1,1) using `writer`TCell c11 = {1, 1, 15};dw.write(c11);// Write the cell c(1,2) using the `operator <<`TCell c12 = {1, 2, 5};dw << c12;
// Java 5 DDS APIDataWriter<TCell> dw = pub.createDataWriter<TCell>(topic);TCell c11 = new TCell(1, 2, 15);dw.write(c11);
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
• A DataReader with default QoS can be declared as follows:
// ISO C++ DDS APIauto dr = DataReader<TCell>(sub, topic);// Read Samplesauto samples = dr.read();// Do something with itstd::for_each(samples.begin(), samples.end(), do_something);
// Java 5 DDS APIDataReader<TCell> dr = sub.createDataReader<TCell>(topic);
Does DDS “JavaScripts”?
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Topic
topic = new dds.Topic(0, 'Cell', 'com.activefloor.Cell')
var myTopic = new dds.Topic(domainID, topicName, topicType);
myTopic = new dds.Topic(domainID, topicName, topicType)
JavaScript
CoffeeScript
var topic = new dds.Topic(0, 'Cell', 'com.activefloor.Cell');
Example:
Example:
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataWriter
dw = new dds.DataWriter(topic)
cell = {}cell.x = 3cell.y = 5cell.pressure = 3
dw.write(cell)
dw = new dds.DataWriter(topic, qos)
CoffeeScript
Example:
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
DataReader
dr = new dds.DataReader(topic)
dr = new dds.DataReader(topic, qos)
CoffeeScript
Example:
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Binding a DataReader
• A DataReader can be bound to a user provided function that will handle incoming data or to a cache
• Notice, that as you are in control of how data-readers are bound to cache you can be very creative
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Binding to User Function
dr.addListener((s) -> console.log(JSON.stringify(s)))
dr.addListener(f)
CoffeeScript
Example:
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Binding to a Cache
// BindingbindCell = dds.bind((s) -> s.x + “-” + s.y)ccache = new DataCache(historyDepth)bindCell(cache, cdr)
// Working with the Cache: Compute number of active cells
activeCells = ccache.map((c) -> if (c > p0) then 1 else 0).fold(0)((a, c) -> a + c)
cache = new DataCache(historyDepth)bind(keyMapper)(dr, cache)
CoffeeScript
Example:
Demo
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Shapes Application
• The iShapes application is used by DDS vendors to demonstrate some of the basic mechanism as well as product interoperability
• Three Topics
• Circle, Square, Triangle
• One Type
struct ShapeType { string color; long x; long y; long shapesize;};#pragma keylist ShapeType color
Spotted shapes represent subscriptions
Pierced shapes represent publications
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Android Shapes
Data Modeling
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Data Centricity
• DDS is Data Centric, it provides an elegant, efficient and scalable data sharing abstraction
• When designing the architecture of a DDS-based system the Data-Model is the first thing that should be devised
• One of the most common question from DDS novices is how to come up with a good data model. DDS Global Data Space
...
Data Writer
Data Writer
Data Writer
Data Reader
Data Reader
Data Reader
Data Reader
Data Writer
TopicAQoS
TopicBQoS
TopicCQoS
TopicDQoS
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Data Modeling in DDS
• A good starting point for identifying DDS topics is to describe your system in a natural language and then use the following table:
English Language DDS
Common Noun Topic
Proper Noun Topic Instance
Transitive Verb Relationship
Intransitive Verb Attribute Type
Adjective Topic Attribute
Adverb Relationship Attribute
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Relationships
• As in DBMS, relationships can be expressed, depending on cardinality and relation attributes, by:
• Keys and foreign keys
• Relationship Topics
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Normalization
• As in DBMS, Topics should be normalized to improve modularity, efficiency and limit if not avoid anomalies. Typically you should have your Topics in the 3NF
• Notice that the nature of DDS may induce further data normalization to deal with conflicting QoS requirements, such as:
• Mix of frequently and infrequently changing data
• Mix of attributes that have different persistence requirements
Part I Summary & Remarks
Copyrig
ht 2013, PrismTech – A
ll Rights Reserved.
Summary & Remarks
• DDS provides an elegant Data Space abstraction that allows application to share data ubiquitously and efficiently -- the Data Space implementation is fully distributed
• DDS provides primitives for defining classes of information as well as organizing it
• Data Modeling and normalization draws heavily from DBMS theory and practice with some additional considerations w.r.t. the nature of DDS
¥@prismtech
¥@acorsaro
¥youtube.com/opensplicetube ¥slideshare.net/angelo.corsaro
¥ crc@prismtech.com
¥sales@prismtech.com
¥opensplice.com ¥forums.opensplice.org
¥opensplice.org ¥opensplicedds@prismtech.com
:: Connect with Us ::