The Impact of Messaging Standards on Event-Driven Architecture and IoT
Messaging for IoT
-
Upload
virtual-jboss-user-group -
Category
Software
-
view
2.316 -
download
0
Transcript of Messaging for IoT
Messaging for IoTJBUG November 2015
Martyn Taylor, Dejan Bosanac
Presenters
Martyn Taylor Senior Software Engineer at Red Hat Apache ActiveMQ Committer
Mainly working on Apache Artemis Keen interest in IoT
Bosanac Dejan Senior Software Engineer at Red Hat Apache ActiveMQ committer and PMC member Co-author of ActiveMQ in Action
Agenda
IoT messaging basics Tasks and challenges Patterns and protocols
IoT messaging brokers Apache ActiveMQ ActiveMQ Artemis
IoT messaging scaling Vertical and horizontal Qpid Dispatch Router Scalable deployments
IoT Messaging Basics
IoT Topology Messaging tasks Messaging challenges
IoT Topology Big Data
Messaging Infrastructure
Analytics
Enterprise Middleware
Devices
IoT Messaging Infrastructure
Task Provides connectivity between devices and back end systems
Challenges Interoperability Deployment environment High Availability Scalability
IoT connectivity patterns
IoT Communication Patterns Telemetry, Command & Control, Enquiry, Notifications
Protocols and Technologies JMS MQTT AMQP
Telemetry
• Device → Service• One way (push)• Failures tolerable• Lots of small data
Command and Control
• Service → Device• Two way (Req/Resp)• Less, more important data• 1 → 1 and 1 → many
• grouping
Enquiry
• Device → Service• 1 → 1• Two way (Req/Resp)
Notification
• Service → Device• One Way (push)• 1 → Many
• Grouping
Requirements on the transport
Communication from/to devices 1 → 1
Device Addressing 1 → Many
Group Addressing Push Request → Response
IoT Networks
Unreliable Handle network failures
Expensive and Constrained Low network overhead
IoT Devices
Limited Resources Low processing overhead
Large scale and varied Inter-operable
Battery powered Work offline
IoT Environment Challenges
Interoperable Connection Failure
Handling Offline Low network overhead Low Processing overhead
JMS
Addresses majority of requirements 1-1, 1-Many,Grouping,Device address
Queues, Topics, Message Selectors Push Req/Resp
Reply to queues Inter-operable
JVM Connection failures
Durable Messages and Various ACK Modes
JMS
Inter-operability Java Runtime only Protocol vs API
Vendor specific Rich feature set
Complexity on the client Large footprint
Vendor protocols High network overhead
MQTT
OASIS standard (v3.1.1) Created by IBM Light weight
Low network message Simple protocol
Pub / Sub Quality of Service Connection failures Very popular in IoT scenarios
QoS Levels
Overheads
Sending / Receiving messages QoS 0 = 8 bytes QoS 1 = min 12 bytes QoS 2 = min 20 bytes
Clients have small code foot print Simple protocol Burden on the broker
Failures
Reconnects
Subscriptions
Topics Hierarchical addresses
e.g. building/core/floor/2/office/1 Supports wild cards
building/core/floor/# building/core/floor/+/office/+
QoS per topic Outlives a connection
AMQP 1.0
International Standard (ISO/IEC ISO 19464) Binary Protocol Rich feature set:
Conversation multiplexing Advanced flow control Type system QoS Guarantees
Symmetrical message exchange No Broker required
QoS Guarantees
At most once Fire and Forget
At least once Retry
Exactly once 3 Way Ack
Type System
Highly Interoperable Rich Type Set
Primitives integer, long lists, maps
Descriptive types Allow application defined types
Mappings in most languages
Symmetrical Protocol
Not Client -> Broker Broker less message
Client -> Client Client -> Router
Interesting features Smart routing
Flow control & multi channels
Limit producer and consumer flow Each channel (link) can be controlled individually Limit telemetry flow for command messages
Protocols Overview Lots of IoT protocol choices including several not discussed here The right choice depends of the scenario Scenarios may combine protocols
CoAPTarget usecase Long-haul (&
local)Long-haul Local (& long-
haul)Long-haul
Compactness High Medium Highest Verbose
Security SSL SSL DTLS SSL
Flow control No Yes No No
Structured message
No Yes No Yes
Complexity Medium High Low Lowest
IoT Messaging Brokers
Apache ActiveMQ
ActiveMQ Artemis
Apache ActiveMQ
Apache ActiveMQ is a high-performance, scalable messaging broker
Started as an open source JMS broker
Moved to Apache Software Foundation in 2006
Now widely used open source messaging system
Apache ActiveMQ
Multi-Protocol/Multi-Language Support
Invented Stomp protocol in 2006
Now supports AMQP 1.0, MQTT 3.1.1, HTTP, STOMP protocols
Broad range of supported client libraries for all kinds of device hardware
Apache ActiveMQ
Benefits Feature rich Battle-tested in many production environments Good security support Flexible Configurable Embeddable
Apache ActiveMQ
Limitations Broker core getting old Written using synchronous thread-locking model
Limits vertical scalability of the broker Number of threads raise with number of connections
and destinations
ActiveMQ Artemis
Multi Protocol Broker AMQP, MQTT, STOMP, OpenWire, Artemis Core JMS (API)
Started as HornetQ JBoss project in 2009 Embedded WildFly (JBoss AS) JMS messaging
service In 2014 donated to Apache ActiveMQ
Sub project ActiveMQ Artemis. Latest Release 1.1.0
ActiveMQ Artemis: Core
Contains Broker Business logic Core maintains a tight scope
Message Routing Persistence Protocol utility API HA and Scaling
Highly Performance Protocols are plugged in
ActiveMQ Artemis: Architecture
ArtemisCore
Apache Artemis Summary
Great performance due to Reactive Architecture Efficient Append only Journal
Multi - Protocol Broker AMQP, MQTT, STOMP, CORE, OpenWire, JMS
HA and Scalability built in
Apache Artemis Next Steps
Take features from ActiveMQ Enhance OpenWire for ActiveMQ compatibility JDBC Journal implementation More protocols
CoAP Your protocol here….
IoT Messaging Scaling
Broker scaling Vertical Horizontal
Qpid Dispatch Router Scalable Deployment
Broker – Vertical Scaling
Give broker enough resources Reduce thread usage
NIO transport Reactive architecture
Improve monitoring under stress Advisory messages Selective Mbean registration
Broker – Horizontal Scaling
One broker can only do so much Horizontal scaling
Networks of brokers Load balance connections
Limitations All destinations on all brokers Broker network is the bottleneck
Qpid Dispatch Router
Lightweight AMQP 1.0 message router written in C http://qpid.apache.org/components/dispatch-router/
Provides flexible and scalable interconnect between AMQP endpoints
Qpid Dispatch Router
It is not a broker It never owns a message It propagates AMQP transfer, settlement and
disposition frames between endpoints Message based or link based routing
Router/device1
/device2
Qpid Dispatch Router
Deployed in multiple router-broker-endpoint topology Redundant paths
RouterBroker
Router
Qpid Dispatch Router
Cost-based route computation
RouterBroker
Router
Qpid Dispatch Router
Automatic re-routing on failure
RouterBroker
Router
Qpid dispatch router
Benefits Better scaling due to more focused tasks Smart routing can be used to partition the traffic Ideal candidate for gateway into the system
Scalable deployments
Smarter scaling techniques using routers Connections concentration Destinations concentration Destination sharding Smart routing
Scalable deployments
Combination of brokers and routers provides powerful tool box
Brokers should focus on storing messages
Routers should do the rest
Allows for horizontal scaling topologies that can solve IoT challenges
Connection Concentration
Router
Broker
• Challenge: Reduce the number of connections on the broker
Destination Concentration
Router
Broker
/building1/room1
/building1
/building1/room2
/building1/room2
• Challenge: Reduce the number of destinations on the broker
Destination Sharding
Router
Broker
Queue.A
Broker
Queue.B
Challenge: Distribute destinations across brokers
Deployments – Smart Routing
Router
BrokerQoS 0
Challenge: Decrease the load of messages on the broker
Deployments – Smart Routing
Router
BrokerQoS 1
Summary
IoT offers a new set of problems for communication Messaging is a good fit We need new tools and technologies to help
Protocols such as AMQP and MQTT help address some of these problems
ActiveMQ and Artemis are evolving to meet IoT demands
New tools such as Apache Dispatch Router allow us to do some novel things to help scale to really high numbers
Resources
• Slides• http://http://www.slideshare.net/dejanb/messaging-for-iot
• MQTT• Spec: http://www.mqtt.org • Libraries and tutorials: http://www.eclipse.org/paho/
• AMQP• Spec: http://www.amqp.org/ • Libraries and tutorials: http://qpid.apache.org/
• ActiveMQ and Artemis• http://activemq.apache.org/
• Dispatch Router• http://qpid.apache.org/components/dispatch-router/