Making communications across boundaries simple with NServiceBus

PowerPoint Presentation

Yves Goeleven & Sean FeldmanSolution Architects & Azure MVPshttp://particular.net Azure Service Bus & NServiceBusSetting up communication across boundaries

1Setting up communication across network boundaries

Introduction to ServiceInsight for NServiceBus

NServiceBus & Azure Service BusSecure & reliable communication between software components across network boundaries.

Inside cloud servicesBetween datacentersBetween branch officesHybrid solutionsIoT & Mobile devicesWhen do you need Azure Service Bus ?

2


NServiceBus & Azure Service BusFramework to simplify the development of distributed business and workflow applications..NET native workflow & pub/subSimplicity, reliability, scalability and performanceRuns on premises, in the cloud, or eitherSupports a wide range of messaging transports, deals with transport technology complexity What is NServiceBus for?

Endpoint

EndpointTransportAzure Service BusRabbitMQMSMQSQL ServerASQSQS

3


NServiceBus & Azure Service BusAn Endpoint is a logical entity that communicates with other Endpoints via messaging.

Endpoint typically equates to a process.

Each Endpoint Has an identifying nameContains a collection of Message Handlers and SagasCan be deployed to a number of machines and environments (instances)

NServiceBus simple overview

Endpoint

EndpointTransportclass PlaceOrder : ICommand{ public Guid Id { get; set; } public string Product { get; set; } }instance.Send(new PlaceOrder())class Handler : IHandleMessages {Task Handle(PlaceOrder message){// your code here} }

4

Demo

https://docs.particular.net/samples/azure/azure-service-bus/

5


NServiceBus & Azure Service BusTransport => MessagePump + Dispatcher ( + additional features)Forked pipeline of behaviors (handler returns)Some behaviors might invoke transactional resources like Persisters Startup infrastructure (TransportDefinition, Feature, Installer, ...)NServiceBus - endpoint internals

MessagePump

DispatcherHandler

Startup Infrastructure

6

Connectivity

7


NServiceBus & Azure Service BusInstaller called at startupTopologyCreator inquires Topology for what needs to be createdFor each entity in the topology constructs creator objectWhich in turn uses namespace manager to create entities deals with all possible exceptionsCreation InstallerTopologyTopologyCreatorWhat needs to be created?NamespaceManager

Queue, Topic & Subscription CreatorsSubscriptionManager

8


NServiceBus & Azure Service BusMessagePump & Dispatched inquire Topology to know what to receive from/send toPools of receivers, senders & factories (default poolsize = #cores)Active lifecycle management (connection reestablished on drop)Circuit breaker patternConnectivity Management

MessagePumpDispatcher

Pool of receiversPool of sendersPool of factoriesSBMP over TCPSBMP over TCPContextLifecycleManagerLifecycleManagerLifecycleManager

Circuit breakerTopology

9


NServiceBus & Azure Service BusRepresents the layout of service bus entities that allow endpoints to communicate.Explicit concept in transport so that it can be swapped.ASBs forwarding feature provides near endless possibilities.

What is a topology?

ForwardingEndpoint CEndpoint A

Endpoint BTopology

10

Lets have a look at some topologies

11


NServiceBus & Azure Service BusEntity usageCommands vs EventsEmbed message types (f.e. filters, entity names)Does each namespace type support the entities usedNative capabilitiesOwnershipEndpoints may not be aware of the full topologyAwareness creates coupling, do you want that?Some entities may not belong to any endpointPerformance impact(Backwards) compatabilityMultiple namespaces, Partitioning, HANo forwarding between namespaces

Things to consider for a topology

12


NServiceBus & Azure Service BusEach endpoint has its own input queue

Sends direct to input queue, does not leverage pub/sub capabilities of ASB, instead uses storage driven publishing from NSBBasic topology

Endpoint C

Endpoint B

Endpoint A

Sphere of ownership

13


NServiceBus & Azure Service BusBasic topology: pros and consProsConsSimple modeNo native pub/subCan leverage any messaging namespace type (Basic, standard, premium)Publisher coupled to subscriber (in both directions)

Easy to port from MSMQExternal dependency on storageIntegrates with auto-scaleNo native auditing on commands/events

14


NServiceBus & Azure Service BusEach endpoint has its own input queue & topic

Sends direct to input queue, publish via topicEndpoint oriented topology

Endpoint C

Endpoint B

Endpoint A

EndpointC.TypeAEndpointB.TypeAEndpointB.TypeBEndpointA.Events

15


NServiceBus & Azure Service BusEndpoint oriented topology: pros and consProsConsPublisher decoupled from subscribersSubscribers coupled to PublishersNo dependency on storageHard to use polymorphic eventsCannot handle changes in event hierarchyPossible event overflowIntegrates with auto-scaleAuto-scale for input queue onlyCan not leverage basic namespaceNative auditing on events Native auditing on events very hardNo native auditing on commands

16


NServiceBus & Azure Service BusEach endpoint has its own input queue, shared topics for publishing

Sends direct to input queue, publish via topicForwarding topology

Endpoint C

Endpoint B

Endpoint A

Type AType A Type BEndpointBEndpointCBundle-1

17


NServiceBus & Azure Service BusForwarding topology: pros and consProsConsNo dependency on storageCan not leverage basic namespaceNo coupling between publisher & subscriberNo native auditing on commandsSupports polymorphismChange in message hierarchy is transparentIntegrates with auto-scaleProtected against overflowNative auditing on events

18


NServiceBus & Azure Service BusEach endpoint has its own input topic, shared topics for publishing

Sends direct to input topic, publish via topicTopic all the things topologyEndpoint CEndpoint BEndpoint A

Type AType A Type B

Bundle-1EndpointBEndpointC

19


NServiceBus & Azure Service BusTopic all the things topology: pros and consProsConsNo dependency on storageCan not leverage basic namespaceNo coupling between publisher & subscriberDoes not integrate with auto-scale

Supports polymorphismChange in message hierarchy is transparentProtected against overflowNative auditing on commands & events

20

Reliable Message Exchange: Receiving

21


Azure Service BusTalk about defaults for connectivity mode, transport type, concurrency, number of clients,

Talk about how nsb dispatches received message (based on type in header)

Talk about how nsbs transactionmode translates to receivemode (and not to transactionscope)ReceivingOverall: how we receive messages and how we decide on what handles are getting invoked.

Cover things like transport encoding (headers, stream/byte[], deserialization, handler dispatching/polymorphism, etc.)

22


NServiceBus & Azure Service Bus1. Unreliable TXs disabled2. Receive Only transport TXs3. Sends atomic with Receive4. Transaction Scope Distributed TXsNSB Transaction Modes

23


NServiceBus & Azure Service BusReceiveAndDelete receive mode

Not recommended as when failures occur, messages will be permanently lost.

Use case: telemetryTransaction Mode: Unreliable

Endpoint

Receive And Delete

Delete

24


NServiceBus & Azure Service BusPeekLock receive mode

Partial results Updates to data storesMessage sends

Higher performanceHA multiple namespacesTransaction Mode: Receive Only

Endpoint

DLQ

Temporal Lock

Delete

25

Endpoint


NServiceBus & Azure Service BusPeekLock receive mode + transaction + send via

No ghost messagesBUT, data is not included in transport transaction (data should have its own transaction)Transaction Mode: Sends atomic with Receive

MessageReceiver

DLQ

Temporal Lock

DeleteMessage Sender

26


NServiceBus & Azure Service BusSend via requires transaction scopeThat scope does not allow other transactional resources inside itRequires strategic suppress scopes in the pipeline to workTransaction Mode: Sends atomic with Receive

MessagePump

DispatcherHandler

Via scopewraps pipeline invocationSuppress scopewraps Handler scopeConnections enlists in Handler scopeEnlists in Via scopeConnections must be wrapped in suppress scope

27


NServiceBus & Azure Service BusDTC is not supported by ASBTransaction Mode: Transaction Scope

28


NServiceBus & Azure Service Bus- Successfully processed messages go to Audit queue

- Recoverability for failed messages - Immediate retries - Delayed retries

Messages failed all retries go to Error queue

Total Number of Attempts Business errors(ImmediateRetries:NumberOfRetries + 1) x (DelayedRetries:NumberOfRetries + 1)

29


NServiceBus & Azure Service BusPoisonous messages are dead lettered w/o retries

Messages with number of immediate retries bigger than entity`s delivery will dead letter.Poisonous & dead-lettered messagesExample:Immediate retries set to XEntity`s DeliveryCount set to X-1Unsupported Transport encoding (i.e. not stream or byte[])

30

Reliable Message Exchange: Sending

31


NServiceBus & Azure Service BusDefault dispatch mode: batchedDispatch Mode

Endpoint AEndpoint BNamespace

Batch

Endpoint AEndpoint BNamespace

BatchOptional (explicit): immediate12

32


NServiceBus & Azure Service BusEach namespace is sent separatelyDispatching to multiple namespacesEndpoint BNamespace 1Endpoint CNamespace 2

BatchBatch

Endpoint A

Batch

33


NServiceBus & Azure Service BusBatch size is estimated using padding percentage. Default padding: 5%.

Calculated message size = Raw body size + Custom headers size (keys + values) + Standard properties estimated size + padding %Batch SizeBody SizeCustom Headers (keys + values)Standard Properties(estimated)Padding %Body SizeCustom Headers (keys + values)Standard Properties(estimated)Padding %

Body SizeCustom Headers (keys + values)Standard Properties(estimated)Padding %Body SizeCustom Headers (keys + values)Standard Properties(estimated)Padding %

Up to MAX_SIZE or 100 messages

34


NServiceBus & Azure Service BusDefaults & what needs to be considered if really high perf is requiredAwaiting for combined Tasks vs individual tasksNumber of Factories and clients, concurrency, prefetchPerformance Tuning

35

Demo

https://docs.particular.net/samples/azure/performance-tuning-asb/

36

Thank you

37

Q&A

38


Azure Service BusNeed to set up secure communication between your software components across network boundaries?

Yves Goeleven and Sean Feldman will show you what it takes to use the Azure Service Bus for this purpose.

Learn how to:Set up and maintain the connection with the broker.Ensure reliable message exchange.Optimize performance.Abstract.

39

Making communications across boundaries simple with NServiceBus

Software

Transcript of Making communications across boundaries simple with NServiceBus