Messaging

○ Messaging models○ Protocols & Standards○ Message brokers○ Data pipeline architecture○ Microservices architecture

Agenda

Overview

Messaging models

Publisher-Subscriber

Point-to-Point

○ AMQP - Advanced Message Queuing Protocol○ MQTT - Message Queuing Telemetry Transport

(machine-to-machine (M2M)/"Internet of Things" connectivity protocol)

○ STOMP - Simple (or Streaming) Text Orientated Messaging Protocol

○ JMS - Java Message Service

Messaging protocols & standards

RabbitMQ○ You have messages that need

to be routed in complex ways to consumers

○ You want per-message delivery guarantees

○ You don't care about ordered delivery

Message broker examplesApache Kafka

○ You have a fire hose of events

○ You need delivered in partitioned order 'at least once'

○ You want to be able to re-read messages

https://www.quora.com/What-are-the-differences-between-Apache-Kafka-and-RabbitMQhttp://queues.io

Pipeline is a set of data processing elements connected into series, where the output of one element is the input of the next one.

Data pipeline architecture

Microservices architecture

Apache Kafka Hits 1.1 Trillion Messages Per Day – Joins the 4 Comma Club

Apache Kafka at LinkedIn

Apache Kafka technical overview

Apache Kafka technical overview

Questions?