itcampro@ itcamp13# Premium conference on Microsoft technologies

Messaging Patterns in the Cloud

Radu Vunvulea

iQuest Group@RaduVunvulea

http://vunvulearadu.blogspot.com

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudHuge thanks to our sponsors!

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• World of Messages

• On-Premise and Cloud solutions

• Messaging Patterns

• Costs, Benefits and Limitations

Agenda

itcampro@ itcamp13# Premium conference on Microsoft technologies

WORLD OF MESSAGES

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudWorld of Messages

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudWorld of Messages

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudWorld of Messages

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudWorld of Messages

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudWorld of Messages

Message

Message

Message

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

Interaction and communication between

software applications

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudEnterprise Service Bus

itcampro@ itcamp13# Premium conference on Microsoft technologies

ON-PREMISE AND CLOUD SOLUTIONS

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Pro

– Open Source

– Flexible

– Good price

– Transactional queue read/write

– Flexible load distribution

• Cons

– No queue management

– Document not up to date

– Only one queue technology supported (MSMQ)

NService Bus

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Pro

– Free for commercial use

– Supports MSMQ and RabittMQ

– Open Source

– Good administration console

• Cons

– No commercial support

– Not as fast as NServiceBus

MassTransit

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Pro

– Scalable

– Lots of features

– Good administration console

– Support

– Cross platform and multiple environments

• Cons

– Is not free

– Latency

Azure Service Bus

itcampro@ itcamp13# Premium conference on Microsoft technologies

MESSAGING PATTERNS

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Filter

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Filter

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Custom rules based on message attributes

• Add attributes to message

Message Filter

BrokeredMessage msg = new BrokeredMessage(messageContent);

msg.Properties["colorCode"] = 1;

• Create subscription with custom rule

RuleDescription rule = new RuleDescription()

{

// 1 - green, 2 - orange

Filter = new SqlFilter("colorCode = 1");

}

namespaceManager.CreateSubscription(

"itcamp", "greensubscription", rule);

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Splitter

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Splitter

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Group messages based on session

Message Splitter

MessageSession msgSession =

msgReceiver.AcceptMessageSession();

BrokeredMessage message;

while ((message = msgSession.Receive()) != null)

{

// Process message

}

msgSession.Complete()

BrokeredMessage message = new BrokereMessage(content);

message.SessionId = "sessionId";

• Consume messages based on session id

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Group messages based on session

Message Splitter

MessageSession msgSession =

msgReceiver.AcceptMessageSession();

BrokeredMessage message;

while ((message = msgSession.Receive()) != null)

{

// Process message

}

msgSession.Complete()

BrokeredMessage message = new BrokereMessage(content);

message.SessionId = "sessionId";

• Consume messages based on session id

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Aggregator

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Aggregator

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Group messages based on session

Message Aggregator

BrokeredMessage message = new BrokereMessage(content);

message.SessionId = "sessionId";

• Use correlation id and filters

BrokereMessage message = new BrokereMessage(content);

message.CorrelationId = “CJ"

topic.Send(message);

namespaceManager.CreateSubscription(

“itcamp”,

“iquestsubscriptioncar”,

new CorrelationFilterExpression(“CJ"));

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Resequencer

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Resequencer

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Resequencer

Topic

Subscriptions

1234

1234

1234

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Resequencer

Topic

Subscriptions

1234

1234

3412

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Session, message index and message count

Message Resequencer

• Mark message as dead letter when in

incorrect order

• Iterate dead letter queue to access a

message with a lower index

BrokeredMessage message = new BrokereMessage(content);

message.Properties["index"] = 2;

message.Properties["count"] = 10

message.SessionId = “CJ-15-IQU";

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Resequencer

MessageSession messageSession =

queueClient.AcceptMessageSession();

int currentIndex = 1;

while(true)

{

BrokeredMessage message = messageSession.Receive();

if(int.Parse(message.Properties[“index”]) != currentIndex)

{

message.DeadLetter();

continue;

}

…

message.Complete();

if(int.Parse(messsage[“count”]) == currentIndex)

{

break;

}

currentIndex++;

}

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudMessage Recipient

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Use a property to specific the list of groups

Message Recipient

• Define custom filters using LIKE operator

BrokeredMessage message = new BrokeredMessage(content);

message.Properties[“groups”] = “orange magenta”;

SqlFilter filter = new SqlFilter(

“groups LIKE ‘%orange%’”);

topic.AddSubscription(“subscription3”, filter);

• OR different properties for each group type

SqlFilter filter = new SqlFilter(

“EXISTS orange”);

topic.AddSubscription(“subscription3”, filter);

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Use a property to specific the list of groups

Message Recipient

• Define custom filters using LIKE operator

BrokeredMessage message = new BrokeredMessage(content);

message.Properties[“groups”] = “orange magenta”;

SqlFilter filter = new SqlFilter(

“groups LIKE ‘%orange%’”);

topic.AddSubscription(“subscription3”, filter);

• OR different properties for each group type

• Use only one property and a prime number

for each group

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudContent-Based Router

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudContent-Based Router

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudScatter-Gather

Topic

Subscriptions

Queue

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudScatter-Gather

Topic

Subscriptions

Queue

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudDynamic Router

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudDynamic Router

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public CloudDynamic Router

Topic

Subscriptions

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• [key, values] properties

• Store properties

• Decorate properties

Dynamic Router

itcampro@ itcamp13# Premium conference on Microsoft technologies

COSTS, BENEFITS AND LIMITATIONS

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• What are the costs of processing:

– 24kB message size

– 10M messages

– 1 Topic

- 8 hours

Costs

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• 10$ Sending

• 27.46$ Bandwidth (sending)

Costs

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• 10$ Sending

• 27.46$ Bandwidth (sending)

• 10 $ Receiving

• 0 $ Bandwidth (receiving)

Costs

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• 10$ Sending

• 27.46$ Bandwidth (sending)

• 10 $ Receiving

• 0 $ Bandwidth (receiving)

• 47.46$ Costs related to Service Bus

Costs

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• 10$ Sending

• 27.46$ Bandwidth (sending)

• 10 $ Receiving

• 0 $ Bandwidth (receiving)

• 47.46$ Costs related to Service Bus

• 8.64$ 4 Medium Worker Roles used to

consume messages

• 56.1$ Total cost

Costs

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Cheap

• 99.9% Uptime

• Extremely scalable and flexible

• No message is lost

• Filters and actions support

• REST API

• Death-letter and transaction support

• Same API for on-premise and cloud

Benefits

itcampro@ itcamp13# Premium conference on Microsoft technologies

Private &

Public Cloud

• Not for real-time application

• When processing more than 1M messages

on the same topic in a 30 minutes time

interval latency increases

• You need to pay for each send/receive

command

• Batch maximum size – 100 messages

Limitations

itcampro@ itcamp13# Premium conference on Microsoft technologies

Q & A

itcampro@ itcamp13# Premium conference on Microsoft technologies

THANK YOU