Introduction to ActiveMQ Apollo

1 Copyright © 2011 Progress So3ware Corpora6on and/or its subsidiaries or affiliates. All rights reserved. A Progress So3ware Company FuseSource Confiden6al

A Progress So3ware Company

Introduc6on to Ac6veMQ Apollo

1

Bosanac Dejan May 2011

2 Copyright © 2011 Progress So3ware Corpora6on and/or its subsidiaries or affiliates. All rights reserved. A Progress So3ware Company

About me

 Bosanac Dejan  Senior So3ware Engineer at FUSESource ‐ hNp://fusesource.com  Apache Ac6veMQ commiNer and PMC member  Co‐author of Ac6veMQ in Ac6on


What we are going to cover?

 Why Apollo?   HawtDispatch   Connec6vity

•  Stomp 1.1, MQTT, JMS, ...

  LevelDB Store   Features

•  REST Based Management

  Future



Why Apollo?

4


Apache Apollo

  An experiment to beNer u6lize high core counts on modern processors

  A completely new broker core that is much more determinis6c, stable, and scaleable

  Branched as a new project and a chance for a clean start

Goal

Resulted


Apollo Architecture

  Reactor Based Thread Model   Scala implementa6on   Protocol Agnos6c   REST Based Management



HawtDispatch

7


HawtDispatch ‐ Introduc6on

  Small (less than 100k) thread pooling and NIO event no6fica6on framework API

  hNp://hawtdispatch.fusesource.org/   Java clone of Grand Central Dispatch   Avoid explicit usage of threads and synchroniza6on points in mul6threaded applica6ons

  Applica6on developer submit tasks to dispatch queues   Fixed‐sized thread pool execute tasks


HawtDispatch ‐ Dispatch Queues

  Global Dispatch Queue •  3 global queues shared • Non determinis6c order •  3 priori6es

  Serial Dispatch Queue •  Serial FIFO queues •  Synchronize certain task execu6ons

DispatchQueue queue = getGlobalQueue(HIGH);

DispatchQueue queue = createQueue("My queue");


HawtDispatch ‐ Submijng Tasks

  Java

  Scala

  Tasks •  non‐blocking •  lock‐free • wait‐free

queue.execute(new Runnable(){ public void run() { System.out.println("Hi!"); } });

queue { System.out.println("Hi!"); } // or queue.execute(^{ System.out.println("Hi!"); })


HawtDispatch ‐ Dispatch Sources

  Trigger a task execu6on based on external event   Ideal for dealing with NIO events

SelectableChannel channel = ... DispatchQueue queue = createQueue() DispatchSource source = createSource(channel, OP_READ, queue); source.setEventHandler(new Runnable(){ public void run() { ByteBuffer buffer = ByteBuffer.allocate(1024); int count; while( (c=channel.read(buffer)) > 0 ) { // just dump it to the console System.out.write(buffer.array(), buffer.offset(), buffer.position()); } } }); source.resume();


HawtDispatch ‐ conclusion

  Ideal for developing highly concurrent applica6ons   Ideal for handling NIO events   Ideal for implemen6ng broker cores   Impressive performances

  New development paradigm means it couldn’t be fiNed into exis6ng Ac6veMQ broker



Connec6vity

13


Message protocols ‐ Overview

  Broker Core is Protocol Agnos6c   Protocols are Plugins

•  STOMP 1.0/1.1 • MQTT v3.1 • Openwire

  Transports are Pluggable too. •  TCP, WebSockets etc.

  Protocol detec6on (all protocols can use 1 port)


Feature: Mul6ple Transports

  TCP   SSL  WebSockets   Secure WebSockets   UDP


Stomp ‐ basics

  hNp://stomp.github.com/   Simple Text Orientated Messaging Protocol   HTTP for the messaging realm   Very simple, so it’s easy to write clients and servers in prac6cally any language

  A lot of exis6ng clients in C, Ruby, Pyhton, JS, PHP, etc.   Provides a way to connect different plamorms in asynchronous way   Also Implemented by Ac6veMQ, RabbitMQ, HornetQ, and many others.


Stomp ‐ Nutshell

  Based on text frames, similar to HTTP ones

  Can transport binary bodies   Frame command for every messaging concept, like CONNECT, MESSAGE, SUBSCRIBE, ACK, etc.

  New in version 1.1 •  Protocol nego6a6on • Heartbeats • NACK • Virtual hosts

MESSAGE subscription:0 message-id:007 destination:/queue/a content-type:text/plain hello queue a^@


Apollo and Stomp

  Both 1.0 and 1.1 Supported   Des6na6on Types

• Queues ‐ /queue/a •  Topics ‐ /topic/b• Durable Subscrip6ons ‐ /dsub/c

  Reliable messaging

SEND destination:/queue/a receipt:001 persistent: true hello queue a ^@ RECEIPT receipt-id:001 ^@


Apollo and Stomp

 More messaging features • Message expira6on •  Topic retained messages •  Topic durable subscrip6ons • Queue browsing • Queue message sequences •  Exclusive subscrip6ons •  Temporary des6na6ons • Des6na6on wildcards •  Composite des6na6ons • Message selectors


Connec6vity ‐ MQTT

  Get at hNps://github.com/fusesource/fuse‐extra/   Focused on:

•  low bandwidth networks •  unreliable networks •  Small footprint / Embedded Devices

  3 QoS Op6ons   Also Implemented by WebsphereMQ, MosquiNo and others.


Connec6vity – JMS API

  StompJMS is a JMS 1.1 client implemented using the STOMP protocol.

  Get it at hNps://github.com/fusesource/stompjms   Client implemented with HawtDispatch

•  Constant number of threads no maNer how many client connec6ons are established.

import org.fusesource.stomp.jms.*;import javax.jms.*;StompJmsConnectionFactory factory = new StompJmsConnectionFactory();factory.setBrokerURI("tcp://localhost:61613”);Connection connection = factory.createConnection(“admin”, “password”);Destination example = new StompJmsDestination(“/queue/example”);


Connec6vity – OpenWire

 OpenWire is the na6ve binary protocol implemented by Ac6veMQ   API op6ons:

•  JMS 1.1 Client of Ac6veMQ 5.x • NMS Client for C# Apps •  CMS Client for C++ Apps

 Working Features • Queues, Topics, Durable Subscrip6ons •  Temporary Des6na6ons •  Transac6ons

  Not Yet Supported •  XA Transac6ons (distributed transac6ons) • Network of Brokers style clustering •  0 sized consumer prefetches



LevelDB Store

23


Message Store ‐ overview

 Message Stores are Plugins   Ships with 2 Op6ons

•  LevelDB Store •  BDB Store

  Used to store •  persistent messages •  non‐persistent messages that needs to be swapped out of memory

  Non‐persistent messages that get swapped out do not get dropped on restart

  Delayed Writes


LevelDB ‐ Basics

 What is LevelDB •  LevelDB is a fast key‐value storage library • WriNen at Google •  Provides an ordered mapping from string keys to string values •  Based on SSTable and Log Structured Merge (LSM) Trees


LevelDB ‐ Basics

  Designed to efficiently store large numbers of key‐value pairs while op6mizing for high throughput, sequen6al read/write workloads

  Ideal for implemen6ng message store index  Much beNer performance over tradi6onal B‐Tree indexes (used in KahaDB)


LevelDB ‐ Basics

  It’s a C++ library (no client‐server support)   Batching writes   Forward and backward itera6on over data   Uses Snappy compression


LevelDB Store

  Uses LevelDB to maintain indexes into log files holding the messages

  Uses a JNI driver on Linux and OS X, but falls back to a pure Java version on other plamorms

  Supports replica6on to get High Availability   Default store in Apollo. Available in Apache Ac6veMQ 5.6.0


LevelDB vs KahaDB

  It maintains fewer index entries per message than KahaDB which means it has a higher persistent throughput.

  Faster recovery when a broker restarts   LevelDB based index provide a much beNer performance than the B‐Tree for sequen6al access

  LevelDB indexes support concurrent read access   Pauseless data log file garbage collec6on cycles


LevelDB vs KahaDB

  Uses fewer read IO opera6ons to load stored messages.   It will only journal the payload of the message once   Exposes it's status via JMX for monitoring   Supports replica6on to get High Availability


LevelDB store ‐ HA

  HA version of store works with Hadoop based file systems  Message log is mirrored to HDFS   It can sync on HDFS file instead of local file system   LevelDB indexes are immutable on disk (SSTables)  On checkpoint .sst files are uploaded to HDFS


LevelDB store ‐ HA Recovery

 On master failure, slave will download • message log •  .sst files associated with the latest uploaded index

  Con6nue with regular recovery process


LevelDB store – HA locking

 Master elec6on is done externally  Mul6ple brokers should never use the same HDFS path   Apache ZooKeeper good op6on for implemen6ng distributed locking

  FuseFabric (hNp://fuse.fusesource.org/fabric/) can be used as well


BDB store

 Not ASL 2.0! You have to Agree to the BDB license & download from Oracle.

 Pure Java implementa6on  Very robust  The BDB library supports advanced features like replica6on (not yet exploited)



Features

35


Feature: JAAS Authen6ca6on

 Use any 3rd party JAAS login module  Ships with security enabled by default • Default id/password is admin/password

 File based user and group configura6on  Supports IP address white and black lists.  X509 Cer6ficates  Op6onal guest login support


Feature: Authoriza6on rules

 Fine grained control of who can • admin, monitor, configure, connect, create, destroy, • send, receive, consume

 On broker resources like: • broker, connector, virtual host, topic, queue or durable subscrip6ons

<access_rule allow="bartenders" action="send,consume” kind=”queue topic” id=”BAR.*”/> <access_rule deny="guests" action="consume"/>


Feature: Config Updates

 All Configura6on files are watch and changes are applied at run6me: • Broker config: etc/apollo.xml • JAAS config files like: etc/user.proper6es • Logging config: etc/log4j.proper6es

 No need to restart to apply config changes


Feature: REST Management

curl -u "admin:password" \ http://localhost:61680/broker/virtual-hosts/default.json



Where is it going?

41


Where has it been?


Feature diff vs Ac6veMQ

Missing in Apollo  Networks of Brokers  Priority Support  Message Groups  Message Scheduling  XA Transac6ons  JMX Management API

Only in Apollo  REST Management API  Secure WebSockets  Message Sequences  Con6nuous Queue Browsing  Run6me Config Updates  Per Consumer Store Prefetch


Future

  Start integra6ng it in Ac6veMQ • Apollo should be a new broker engine for 6.0 • We should try to port all exis6ng features (networks, priority queues, XA, etc.)

  Tons of interes6ng work ahead of us ‐ Join us


Ques6ons?

  Links: • Apache Apollo hNp://ac6vemq.apache.org/apollo/ •  STOMP Benchmarks hNp://hiramchirino.com/stomp‐benchmark/ • MQTT Protocol Plugin for Apollo hNps://github.com/fusesource/fuse‐extra • HawtDispatch hNp://hawtdispatch.fusesource.org/ •  StompJMS hNps://github.com/fusesource/stompjms

  Blog: hNp://www.nighNale.net

  TwiNer: •  hNp://twiNer.com/dejanb •  hNp://twiNer.com/fusenews

Introduction to ActiveMQ Apollo

Technology

Transcript of Introduction to ActiveMQ Apollo