Distributed Systems Architecture in Software Engineering SE11
Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems...
Transcript of Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems...
![Page 1: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/1.jpg)
ni.com
Data Communication Methods for Distributed Systems
Lou Kwok How
KEC, Technical Director
![Page 2: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/2.jpg)
Company Overview
• KEC is a systems integrations company that specializes in National
Instruments LabVIEW systems software development and hardware
integration with more than 10 years of experience.
• Area of Expertise:
• NI LabVIEW application/system software development• Ranging from single-tasked applications to multi-device multi-network system
software design/development
• Integration with NI hardware as well as 3rd party devices
• Consultation services for hardware selection and LabVIEW software
design• Working alongside customers’ engineering team to solve their challenges
• Provide solutions to customers that caters to and adhere to their processes and
requirements
![Page 3: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/3.jpg)
Session Goals
▪ Introduce common & scalable communication architectures
▪ Understand what challenges each communication architecture solves
▪ Identify which protocols are ideally suited for different architectures
▪ Example application architectures
![Page 4: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/4.jpg)
Why is Scalability Important?
(Client, Host, etc.) (Node, Target, etc.)
![Page 5: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/5.jpg)
Why is Scalability Important?
![Page 6: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/6.jpg)
Why is Scalability Important?
![Page 7: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/7.jpg)
Goal of Scalability
Minimize the effort and risk associated with adding “just one more”
![Page 8: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/8.jpg)
Strategies for Scalability
▪ Use flexible communication architectures that require minimal foreknowledge
of nodes in the system
▪ Use communication protocols that are optimized for the type of data being sent
& received
▪ Design the system so data communications has little impact on the system
performance and critical operations
![Page 9: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/9.jpg)
Types of Data
Tag
Message/Command
Stream
Current value data
Intermittent data, reliable delivery
Continuous flow, high throughput
![Page 10: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/10.jpg)
Protocols / Mechanisms
Tag
• UDP
• PSP
• Shared Variables
Message
• TCP
• Network Streams
• Web Services (REST)
• Others (DDS)
Stream
• TCP
• Network Streams
• File Transfer
![Page 11: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/11.jpg)
Single Node Architecture
▪ Communication – 1 Client : 1 Node
▪ Focus on Scalability – Low / None
▪ Functionality is typically primary concern
▪ Applications
▪ Simple control & monitoring
▪ Prototyping
▪ Potential Concerns
▪ Easy replacement of either client or node
![Page 12: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/12.jpg)
System Replication
![Page 13: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/13.jpg)
Multi-Node Architecture
▪ Communication – 1 Client : N Nodes
▪ Focus on Scalability – Client code
▪ Nodes added by simple replication, client should adapt to multiple targets with little effort
▪ Applications▪ Distributed control & monitoring (small scale)
▪ Potential Concerns▪ Synchronization of node data
▪ Growing # of unique network streams
![Page 14: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/14.jpg)
Shared Variable Static Nodes vs. Programmatic APIs
▪ (Node) Scalable systems ONLY use static nodes local to the library location
▪ Easy code readability, eliminates need for more complex API
▪ (Client) Programmatic API allows re-use of same code to read/write variables
on multiple nodes
▪ Provide unique IP addresses to different instances of the same code
![Page 15: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/15.jpg)
Shared Variable Static Nodes vs. Programmatic APIs
![Page 16: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/16.jpg)
Human Error on adding nodes
![Page 17: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/17.jpg)
Human Error on adding nodes
![Page 18: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/18.jpg)
Timestamp Synchronization (IEEE-1588/Precision Time Protocol)
![Page 19: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/19.jpg)
Time Sensitive Networking (IEEE 802.1AS)
IEEE 802.1AS-compliant Switch
![Page 20: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/20.jpg)
Server-less Architecture
▪ Communication – N Clients : N Nodes
▪ Focus on Scalability – Client & Node code
▪ Client should adapt add’tl Nodes w/ little effort
▪ Usually implemented with Shared Variables
▪ Applications▪ Multiple clients for simple distributed system
▪ Any client can talk to any node
▪ Potential Concerns▪ Load on nodes due to multiple client access
![Page 21: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/21.jpg)
Shared Variables - The Sweet Spot
▪ Monitoring – Current value (tag) data
▪ If you can’t afford to lose some data, don’t use Shared Variables
▪ Control – Non critical / non time-dependent settings
▪ If a malfunction or safety issue can be caused, don’t use Shared Variables
▪ Slow Data Rates (< 10 Hz)
▪ If you need to go faster, don’t use Shared Variables
![Page 22: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/22.jpg)
Scalability Red Flags with Shared Variables
Shared Variable Updates per Second = # of Shared Variables * Update Rate
(Hz)
The relationship between CPU load and shared variable updates per second are shown for each
target in the following figures.
![Page 23: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/23.jpg)
Shared Variables – Scalability Tips
▪ Update variables only when needed
▪ Update variables at different rates in different loops
▪ Mass updates of variables puts unnecessary load on system
▪ Subscribe only to the variables you need
▪ If you only need to read a few variables, don’t subscribe to all of them
▪ Use small, well organized libraries
▪ It’s easier to deploy, update, subscribe and read variables like this
▪ ALWAYS benchmark your worst case scenario
▪ There are many factors that affect system performance with variables
![Page 24: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/24.jpg)
Single Server Architecture▪ Communication – N Clients : N Nodes
▪ Focus on Scalability – Server code
▪ Nodes can be added by replicating & configuring server
▪ Any client can connect without the server having prior knowledge of it
▪ Applications▪ Multiple client nodes for complex
distributed system
▪ Any client can talk to any node
▪ Potential Concerns▪ Load on server due to multiple client
access
![Page 25: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/25.jpg)
Offline Waveform Processing
Advantages
▪ Eliminates unnecessary constant streaming of
data to clients
▪ Utilizing non-volatile storage on the server frees
up disk space on the nodes
▪ Server can use NI solutions like Data Finder
Server Edition or 3rd party solutions to
aggregate data from many nodes
Concerns
▪ Network bandwidth sending large volumes of
data from many nodes to server
![Page 26: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/26.jpg)
Message BrokerAdvantages
▪ Allows any client losslessly send messages or receive a response from any node while minimizing # of communication channels
▪ Server is responsible for keeping track of where nodes are on the network
▪ See Advanced Message Queuing Protocol (AMQP) and applications like RabbitMQ (or LabbitMQ for LabVIEW)
Concerns
▪ Scalability burden entirely on server
▪ Creating functional and scalable server code can be challenging
▪ Multiple clients simultaneously accessing same node
![Page 27: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/27.jpg)
STM toolkit
▪ Reduce complexity for designing a network communication system for
distributed embedded client.
![Page 28: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/28.jpg)
Example: NI SystemLink
Manage distributed systems with web application software for mass software
deployment, device management, and data communications.
▪ Nodes: LabVIEW API for data communication (Tags, Messages, Files)
▪ Clients: Web application, LabVIEW applications, NXG WebVIs, Data Dashboard
▪ Server: Interact with LabVIEW API, Data storage
![Page 29: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/29.jpg)
SystemLink Architecture
Tag
ServiceMessage
Broker
File
Service
SystemLink Server
High performance, secure data
services for tags, messages, and files.
Intuitive LabVIEW and
HTTP APIs
• NI Controllers
• PC-Controlled
Devices
• 3rd Party PC
Nodes
LabVIEW
LabVIEW NXG
C, PYTHON,
OTHER
Data Service APIs
Web
Browser
Live Data Dashboards
LabVIEW NXG WebVIs
![Page 30: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/30.jpg)
Multi-Server Architecture
![Page 31: Data Communication Methods for Distributed Systems · 2018-07-31 · Manage distributed systems with web application software for mass software deployment, device management, and](https://reader034.fdocuments.net/reader034/viewer/2022050519/5fa3106812b5bb033b50f41d/html5/thumbnails/31.jpg)
Summary
▪ Select the right communication mechanism
▪ You probably will need more than one!
▪ Minimize coupling between components
▪ Maintain as few connections as possible, offload routing to servers
▪ Configuration should be easy
▪ Remove the need for configuration or automate it with config files
▪ Minimize unnecessary network traffic
▪ Send deltas when possible, only to whoever NEEDS it
▪ Use timestamp synchronization when appropriate