Scalable Persistent Message Brokering with WSO2 Message Broker
-
Upload
srinath-perera -
Category
Technology
-
view
3.152 -
download
0
Transcript of Scalable Persistent Message Brokering with WSO2 Message Broker
![Page 1: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/1.jpg)
Scalable Persistent Message Brokering with WSO2 Message Broker
Srinath PereraSenior Software ArchitectWSO2 Inc.
![Page 2: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/2.jpg)
Outline• Understanding Messaging
• Scalable Messaging
• WSO2 MB Architecture
• Distributed Pub/Sub architecture
• Distributed Queues architecture
• Usecases
• Conclusion
photo by John Trainoron Flickr http://www.flickr.com/photos/trainor/2902023575/, Licensed under CC
![Page 3: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/3.jpg)
What is Messaging ?• We often program and design
distributed systems with RPC style communication (E.g. Web Services, Thrift, REST)
• RPC communication is
• Request/Response (there is always a response)
• Synchronous (client waits for response)
• Non-persistent (message is lost if something failed)
• But there are other 7 possibilities
![Page 4: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/4.jpg)
Messaging Systems in Real World
• Sensor networks
• Monitoring/ Surveillance
• Business Activity Monitoring
• Job Scheduling Systems
• Social Networks
http://www.flickr.com/photos/imuttoo/4257813689/ by Ian Muttoo, http://www.flickr.com/photos/eastcapital/4554220770/, http://www.flickr.com/photos/patdavid/4619331472/ by Pat David copyright CC, http://www.fotopedia.com/items/flickr-
2548697541
![Page 5: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/5.jpg)
WSO2 Inc. 5
Why Messaging?
![Page 6: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/6.jpg)
Messaging Systems
• Message Broker(s) as the middlemen
• There are two main models
• Queues
• Publish/Subscribe
http://www.geograph.org.uk/photo/2639458 and http://www.geograph.org.uk/photo/1138150
![Page 7: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/7.jpg)
7
Distributed Queues
• A queue in the “Network”
• API Operations
• Put(M) – put a message
• Get() – get a message (dqueue)
• Subscribe() – send me a message when there is one
• E.g. SQS (Amazon Queuing Service)
• Use cases:- Job Queues, Store and process
![Page 8: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/8.jpg)
WSO2 Inc. 8
Publish/ Subscribe
• There is a topic space based on interest groups
• Publishers send messages to brokers
• Subscribers register their interest
• Brokers matches events (messages) and deliver to all interested parties
• Usecases: Surveillance, Monitoring
![Page 9: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/9.jpg)
WSO2 Inc. 9
Messaging APIs and Message Formats and Standards
![Page 10: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/10.jpg)
WSO2 Inc. 10
Scaling Message Brokers
There are several dimensions of Scale Number of messages Number of Queues Size of messages
Scaling Pub/Sub is relatively easy E.g. Narada Broker, Padres
Scaling Distributed Queues is harder
![Page 11: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/11.jpg)
Scaling Distributed Queues
![Page 12: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/12.jpg)
WSO2 Inc. 12
Scaling Distributed Queues (Contd.)
Topology Pros Cons Supporting Systems
Master Salve Support HA No Scalability Qpid, ActiveMQ, RabbitMQ
Queue Distribution Scale to large number of Queues
Does not scale for large number of messages for a queue
RabbitMQ
Cluster Connections Support HA Might not support in-order delivery Logic runs in the client side takes local decisions.
HorentMQ
Broker/Queue Networks Load balancing and distribution
Fair load balancing is hard
ActiveMQ
![Page 13: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/13.jpg)
WSO2 Inc. 13
MB2 Messaging Architecture
![Page 14: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/14.jpg)
WSO2 Inc. 14
WSO2 MB
![Page 15: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/15.jpg)
Cassandra and Zookeeper• Cassandra
• NoSQL Highly scalable new data model (column family)
• Highly scalable (multiple Nodes), available and no Single Point of Failure.
• SQL like query language (from 0.8) and support search through secondary indexes (well no JOINs, Group By etc. ..).
• Tunable consistency and replication
• Very high write throughput and good read throughput. It is pretty fast.
• Zookeeper
• Scalable, fault tolerant distributed coordination framework
![Page 16: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/16.jpg)
How Distributed Queues Works ?
![Page 17: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/17.jpg)
How Distributed Queues Works (Contd..)
![Page 18: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/18.jpg)
How Distributed Queues Works (Contd.) Users can publish to any node (to a topic) When published, the node writes the message to queue in
Cassandra called “global queue” Each node have a queue in Cassandra called the node queue A worker running in a node reads message from global queue
and writes messages to a node queue that has a subscription for that topic.
A worker in each node reads messages from node queue and delivers to subscriber for that queue.
Node deletes messages only when subscriber has acked the delivery
![Page 19: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/19.jpg)
How Pub/Sub Works ?
![Page 20: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/20.jpg)
How Pub/Sub Works (Contd.) ?
![Page 21: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/21.jpg)
Fault Tolerance
• We write the message to Cassandra once we receive the message
• We always read, process and then only delete messages (e.g. at client delivery after receiving the Ack)
• In case of a failure of nodes, then worse case there will be duplicates
Copy right , http://www.fotopedia.com/items/flickr-6206406047:CC license
![Page 22: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/22.jpg)
WSO2 Inc. 22
JMS support for MB2
Feature Yes
Pub / Sub Yes
Durable Subscriptions Yes
Hierarchical Topics Yes
Queues Yes
Message Selectors No, planned for 3.0
Transactions No, planned for 3.0
![Page 23: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/23.jpg)
How does MB2 Make a difference?
• Scale up in all 3 dimensions
• Create only one copy of message while delivery
• High Availability and Fault Tolerance
• Large message transfers in pub/sub (asynchronous style)
• Let users choose between strict and best effort message delivery
• Replication of stored messages in the storagehttp://www.flickr.com/photos/flickcoolpix/
3566848458/sizes/m/in/photostream/
![Page 24: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/24.jpg)
Usecase 1: Store and Process
![Page 25: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/25.jpg)
Usecase 2: Message Bus
![Page 26: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/26.jpg)
Future Work and Roadmap
• WSO2 MB (2013 Q4)
• Support for in-memory delivery through hazelcast (www.hazelcast.com)
• AMQP 1.0 support
• Transaction support
• If you have thoughts, please chat with me or join us at [email protected]
http://www.flickr.com/photos/24071429@N08/2314391179/
![Page 27: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/27.jpg)
Conclusion• Provides an alternative architecture for
scalable message brokers using Cassandra and Zookeeper
• It provides
• A publish/subscribe model that does not need any coordination between broker nodes
• A strict mode for distributed queues that provides in order delivery
• A best-effort mode for distributed queue
http://ambr0.deviantart.com/art/Looking-Back-Wolf-310857819
![Page 28: Scalable Persistent Message Brokering with WSO2 Message Broker](https://reader035.fdocuments.net/reader035/viewer/2022062300/55d537c6bb61eb251b8b45da/html5/thumbnails/28.jpg)
Questions?