MQSERIES Presentation

© 2004 Cognizant Technology Solutions. Proprietary and confidential.

INTRODUCTION TO MIDDLEWARE Websphere MQSeries


MQSeries

Middleware? It’s a software or software components that help heterogeneous

clients to connect to a common server/servers.

MQSeries? It allows independent and potentially non-concurrent applications on a

distributed systems to communicate with each other

Messaging & Queuing? Message queuing enables distributed applications to reliably

exchange data regardless of hardware, operating system, or available connectivity.

Analogous to e- Mail system Links applications across disparate platforms


Messaging & QueuingINTRODUCTION

Messaging & Queuing


Communication Styles


Messaging & Queuing

Messaging - Applications communicate by sending data in messages rather than by calling each other directly.

Queuing - Messages are put on queues in storage, eliminating the need for programs to be logically connected.

Applications doesn’t need to bother about the inherent network communications, which is taken care by the messaging & queuing products.

Messaging & Queuing provides synchronous & asynchronous communications between the applications.

A messaging and queuing framework is inherently ASYNCHRONOUS!


Asynchronous vs. Synchronous Communications

Synchronous: Application sends request, then blocks until request is processed.

Requires service available at EXACTLY same time as client needs service. It supports symmetrical, circuit-switched, point-to-point connections

Asynchronous: Application sends request and checks at some future time if complete.

Service need not be available when client sends request. It supports symmetrical and asymmetrical, packet-switched point-to-

multipoint connections .


Synchronous Communication

Advantages Disadvantages

Easy to programOutcome is known immediatelyError recovery easier (usually)Better real-time response (usually)

Service must be up and readyRequestor blocks, held resources are

“tied up”Usually requires connection-oriented

protocol


Asynchronous Communication

Advantages Disadvantages

Service need not be available when request is made

No blocking, so resources could be freed

Could use connectionless protocol

Response times are unpredictable Error handling usually more

complex


What's in a message?

Application messages also have two parts: 1. The application data (Content of message)2. The message envelope (Message Identifiers)

Two major components that make up the messaging environment are Messaging API Message-queue management services


What is in a Queue?

A queue is an abstract data type where Data is added to the tail of a list and Data retrieved and/or removed from the head of the list.

General Characteristics Queues are described as having First In, First Out (FIFO) behavior,

where the earliest added entries are removed first. Some queues enforce strict FIFO ordering, but the message position

may vary as per their priorities.

Physical nature of Queue A volatile buffer area in the memory of a computer or Data set on a permanent storage device (such as disk)


Queuing

Queuing is the mechanism by which messages are held until an application is ready to process them.

Queuing allows applications to:• Communicate between programs (which may each be running in different

environments) without having to write the communication code.

• Select the order in which a program processes messages.

• Store messages unless being retrieved by receiving application.


Main Features of Messaging & Queuing

There are no direct connections between programs.

Communication between programs can be time-independent.

No Constraints on Application Structure

Communication can be driven by events.

Applications can assign a priority to a message.

Applications Shielded from Environmental Differences

Security.

Data integrity.

Recovery support.



No direct connections between programs

The mechanism by which a message moves from queue to queue is hidden from the programs. Hence the programs are simpler.



Time-independent communication Programs requesting others to do work do not have to wait for the reply to a request.

Either program can be unavailable



There can be a one to many relationship between applications

No Constraints on Application Structure Programs requesting others to do work do not have to wait for the reply to a request.

There can be many to one relationship between applications



Event-driven processing Programs can be controlled according to the state of queues. E.g. Programs

can start as soon as messages arrives on the queue.

Message priority A program can assign a priority to a message when it puts the message on

a queue. This determines the position in the queue at which the new message is added.

Applications Shielded from Environmental Differences Don’t care what OS is used. Don’t care what language they’re written in Don’t care what the underlying communication protocol is used

Security Authorization checks are carried out on each resource like queues

Data integrity Data integrity is provided via units of work.


MQ Series Messaging Platforms


MQ Series Messaging Platforms

MQSeries Messaging Platforms


Websphere MQ Series Basics


MQ Series Basics

MQSeries is a middleware product from IBM that runs on multiple platforms and enables applications to send messages to other applications.

MQSeries is renamed as WebSphere MQ now.

MQSeries takes care of the storage, logging and communications details required to guarantee delivery of the message to

the destination queue.

MQSeries can take care of translating the data when source and target application uses different character sets.

EBCDIC on MVS vs. ASCII on NT or Unix


MQ Series Extended Products

MQ forms the basis of a family of products that include: MQSeries Integrator – A message broker with data transformation and

mapping functions (MQSI – now called WebSphere Business Integration Message Broker)

MQSeries Workflow - Model-driven e-business process automation and tracking

MQSeries Everyplace – Allow mobile workers with laptops, phones and PDA’s to participate in MQ networks.

WebSphere InterChange Server


MQ Series Basics

MQSeries use a common application programming interface known as Message Queue Interface (MQI)

MQSeries also provides a high level message handling API called Application Messaging Interface (AMI) which is a simple application programming interface, providing support for point-to-point and publish/subscribe messaging.

MQSeries supports a wide range of programming languages

ActiveXAssembler (MVS/ESA)C, C++COBOLJavaJMSAMILotusScriptPL/1RPG (AS/400)SmallTalkTAL (Tandem NSK)Visual BasicPowerbuilder


MQSeries Basics - Installation Types

MQSeries Server It’s the Messaging & Queuing software. Provide infrastructure for storing ,

forwarding and maintaining data integrity ,etc for messages

MQSeries Client It’s the software to connect to MQSeries Servers. The communication

between the client and server is synchronous as the client does not have any MQSeries Objects other than set of libraries for applications to connect to MQSeries Servers.


MQ Series – Messaging Patterns

One-to-one or Point-to-Point Each Message is addressed to a specific

Queue Queues retain all messages till they are

consumed or the messages expire

Characteristics Each message has only one consumer. The message consumer can, if required,

acknowledge the successful processing of a message



One-to-many message flows Messages are put onto a queue as before

but can be read from that queue by multiple processes.

For the purposes of load balancing, or simply that the message information needs to be used in different ways

Characteristics More than one possible consumer for a

message Message consumers can, if required,




Many-to-one message flows Messages from multiple processes are

addressed to a single specific queue. Messages sent to them are retained until the

messages are either consumed or until the messages expire.

Characteristics No timing dependencies between the senders

and consumer Message consumer can, if necessary,




Publish/subscribe Model It is a style of messaging application in which the providers of information (publishers) are

decoupled from the consumers of that information (subscribers) using a broker. Message brokers make sure that messages arrive at the correct destinations, and are

transformed to the format required at each destination.



Request/reply Request/reply pattern involves two processes:

• One consists of sending a message and expecting a reply

• Other sends reply messages upon receiving a request message.

Flow of message is complete when the initiating side receives the reply message.

The request/reply pattern can be used with either one-to-one (point-to-point) or publish/subscribe patterns.



One-way datagram, or send-and-forget pattern Sending application sends messages without expecting any reply from the

receiving application. Fully decoupled systems and communication is asynchronous


MQSeries Basics - Messages

Messages are String of Bytes Used to transmit information between programs

Two parts of a Message

Application Data Application determines the content

Message Descriptor/Header Identifies the Message and holds the control information

The message descriptor identifies the message (message ID) and contains control information, also called attributes, such as message type, expiry time, correlation ID, priority, and the name of the queue for the reply.

A message can be up to 4 MB or 100 MB long


MQSeries Basics – MQ Message


MQ Series – Message Descriptors

Version: • The version of the message descriptor depends on the MQSeries version and platform you use.

Message ID:• This is a byte string that is used to distinguish one message from another • Generally, no two messages should have the same message identifier, although this is not disallowed by the queue manager. The message identifier is a permanent property of the message, and persists across restarts of the queue manager.


MQ Series - Message Descriptors

Correlation ID: These fields allow the application that receives the reply to correlate the

reply with its original request.

Persistent/Non-persistent• Persistent messages always arrive at their destination, even when the system

fails.

• “Hardened” - saved on disk

• specific message persistent or all messages on a particular queue

Time and Date: when the MQPUT occurred

Lifetime: When this date is reached and an MQGET is issued, then the message will be discarded

Format: use to decide whether data conversion can be done or not.


MQSeries Basics – MQ Message Types

Message Types – 4 Types1. Datagram

A message containing information for which no response is expected.

2. Request A message for which a reply is requested.

3. Reply A reply to a request message.

4. Report• A message that describes an event such as the occurrence of an error or a

confirmation on arrival or delivery.• Types

• Exception Report Message

• Expiry Report Message

• Confirmation On Arrival (COA)

• Confirmation On Delivery (COD)

• Positive Action Notification (PAN)

• Negative Action Notification (NAN)


MQSeries Basics – MQ Message Types

Message Types – Application View1. Persistent

Delivery of persistent messages is assured. Slower Recovery

2. Non-Persistent Delivery is not assured Faster No Recovery


Websphere MQ Series Objects


MQSeries Basics – MQSeries Objects

Queue managers Queues Namelists Process definitions Channels Listener AuthInfo


MQSeries Basics – Queue Manager

Heart of MQSeries is the message queue manager (MQM).

It’s a MQSeries run-time program

Runtime program - It enables developers to run a development application

program that's written for that application. The software company that owns

the application may require a licensing fee or may allow developers to freely

distribute runtime versions with the development application.

Queue Managers have names (identities) that are UNIQUE in a network (like host names).

Multiple Queue Managers can reside on the same machine MQSeries Objects (like Queues & Channels) belong to Queue

Manager.


MQSeries Basics – Queue Manager Functions

Manages MQSeries Objects and Messages Provides Message Queuing Interface (MQI) for applications to

communicate. Applications invoke functions using API Calls

Controls access to queues:

Administration (create, delete, etc)

Usage (Put, Get)

Applications puts Messages to Queue on a Queue Manager


MQSeries Basics – Queue Manager Functions

Queue manager transfers messages to other queue managers via

channels using existing network facilities, such as TCP/IP, SNA or

SPX. MQM’s require Channels for communication

Serves as transaction coordinator (syncpoint) for all queue operations.

Segments and Assembles the messages if required. Can send one message to more than one destination with one API call

using a user-defined distribution list, thus reducing network traffic.

MQSeries clients do not have a queue manager in their machines. Client machines connect to a queue manager in a server.


MQSeries Basics – Queue Manager control Commands

Control Commands : Control commands to perform operations on queue managers, command servers, and channels.

Create a Queue Manager crtmqm –q (QM1)

Start the Queue Manager strmqm (QM1) (or) strmqm –> to start default QM

Stop the Queue Manager endmqm –i (QM1) c, i and p options

Where (c= Controlled, i= immediate, p=pre-emptive).

Delete the Queue Manager dltmqm (QM1)


MQSeries Basics – Queue Manager control Commands

Queues and Channels are defined in MQSeries command mode

To enter MQSeries command mode runmqsc QM1 (or) runmqsc -> for default QM

Display Queue Manager attributes Display qmgr

Alter Queue Manager attributes Alter qmgr *

Exit MQSeries command mode end


MQSeries Basics - Queues

Message queues are used to store messages sent by programs Types of Queues


MQSeries Basics - Queues1. Local Queue

Queue Owned by Queue Manager to which program is connected Store Messages for programs that use the Same Queue Manager PUT & GET Message The syntax used to create a local queue is given by “DEFINE QLOCAL(XX)” (XX IS THE NAME OF THE LOCAL Q NAME)

2. Remote Queue Queue owned by a different Queue Manager Local Definition of Remote Queue Local Queue Manager forward the message to remote Queue Manager Program cannot read messages from Remote Queue The syntax and the properties needed to be created along with a remote Q is given

by “DEFINE QREMOTE(XY) RNAME(YZ) RQMNAME(ZX) XMITQ(Z1)” (YZ IS THE NAME OF THE Q WHERE THE MESSAGE HAS TO BE

TRANSFERRED,ZX IS THE QMANAGER WHERE THE MESSAGE HAS TO BE TRANSFERRED,Z1 IS THE TRANSMITQ WHICH ACTS LIKE A MEDIATOR TO SEND MESSAGE TO ANOTHER QMANAGER)



3. Transmission Queue Local Queue with special purpose Remote Queue is associated with Transmission Queue Intermediate step when sending message to remote queues Used internally by Queue Managers. The syntax and the property to be created along with a transmitQ is given

by

“DEFINE QLOCAL(X1) USAGE(XMITQ)” (X1 IS THE NAME OF THE LOCAL Q BUT AS WE HAVE GIVEN

USAGE AS XMITQ IT WILL BE USED AS TRANSMITQ)



4. Dynamic Queue Local Queue defined “on fly” when application needs it. Can be retained by Queue Manager or can be automatically deleted. Types:

• Temporary queues – Do not survive Queue Manager restart

• Permanent Queues – Survive Queue Manager restart

5. Model Queue Not a real queue Collection of attributes to create a dynamic queue

6. Alias Queue Are definitions not real queues Used to assign different names to same physical queue



7. Initiation Queue Local Queue used by Queue Manager for triggering purpose Syntax to create is as same as how we create a localQ.

8. Reply-To Queue Name of queue to which reply should be sent. Specified in request message

9. Dead-Letter Queue Queue into which Queue Manager writes the message if the messages cant be written

to queue. Some samples:

• The destination queue is full.

• The destination queue does not exist.

• Message puts have been inhibited on the destination queue.

• The sender is not authorized to use the destination queue.

• The message is too large.

• The message contains a duplicate message sequence number.



10. Repository Queue Used in conjunction with Clustering Holds the cluster information

11. Event queues Local Queues to hold instrumentation events. Messages are generated by Queue Manager Sample instrumentation (events) are

• A queue becoming full.

• A channel being started

12. Command queues Local Queue named SYSTEM.ADMIN.COMMAND.QUEUE For sending MQ Commands Created automatically along with Queue Manager Creation


MQSeries Basics - Namelists

Namelists Contains list of other MQ Objects

• Eg, in Cluster, identify a list of clusters for which the queue manager holds the repository

Typically used to identify group of queues Used with queue manager clusters to maintain a list of clusters referred to

by more than one WebSphere MQ object.


MQSeries Basics - Process definitions

Process definitions Associated with triggering Define an application that starts in response to a trigger event. Process definition attributes include

• application ID

• application type

• data specific to the application

To allow an application to be started without the need for operator intervention the attributes of the application must be known to the queue manager. These attributes are defined in a process definition object. The ProcessName attribute is fixed when the object is created; you can change the others using the WebSphere MQ commands or the WebSphere MQ for z/OS operations and control panels. You can inquire about the values of all the attributes using the MQINQ call.


MQSeries Basics – AuthInfo

AuthInfo: An AUTHINFO object contains authentication information used in Secure

Sockets Layer (SSL) encrypted transport of information. An AUTHINFO object of AUTHTYPE CRLLDAP provides the definitions required to perform Certificate Revocation List (CRL) checking using LDAP servers.


MQSeries Basics - Channels

Channels Provide communication path between Queue Managers Logical communication link Channel Types

1. Message channels• Connects two Queue Managers via Message Channel Agents (MCAs).• Unidirectional• Consists of Sender & Receiver• Transfers messages from XMIT Queue to communication link and from Communication link to

target Queue2. MQI channels

• Transfer MQI calls from a WebSphere MQ client to a queue manager, and responses from a queue manager to a WebSphere MQ client.

• MQI Channel is Bi-Directional

Supported Transport Types

SNA LU 6.2

TCP/IP,

NetBIOS, SPX

DEC Net



Channels SENDER CHANNEL(SDR) RECIEVER CHANNEL(RCVR) SERVER CONNECTION CHANNEL(SVRCON)

SENDER CHANNELS COMMANDS: DEFINE CHANNEL(Channel.name) CHLTYPE(SDR) CONNAME(‘IP

Address(port)’) XMITQ(Transmission Queue name) TRPTYPE(TCP) Creates a Channel with Channel.name And ChannelType is Sender(SDR)

With Connection Name along the Designation port number of the Listener And Defining XMITQ with transmission queue name and Transmit Type is TCP (We can use TCP,SUA etc.).


RECIEVER CHANNELS COMMANDS: DEFINE CHANNEL(Channel.name) CHLTYPE(RCVR)

Creates a Channel with Channel.name And ChannelType is Reciever(RCVR)

NOTE: Sender Channel name and corresponding peer side Reciever Channel name should be same.

SERVER CONNECTION CHANNEL(SVRCON) : DEFINE CHANNEL(Channel.name) CHLTYPE(SVRCON)

Creates a Channel with Channel.name And ChannelType is Server Connection(SVRCON)



Listener: A channel listener is a program to start receiving (responder) MCAs Responder MCAs are started in response to a startup request from the

caller MCA; the channel listener detects incoming network requests and starts the

associated channel.

DEFINE LISTENER(Listener.name) TRPTYPE(TCP) PORT(Port no) Creates a Listener with Listener.name And Transmit Type is TCP And Port

no of the Listener

MQSeries Basics - Listener


MQSeries Basics – Installation & Basic Steps

Install MQSeries

Create Queue Manager crtmqm -q -u SYSYTEM.DEAD.LETTER.QUEUE MYQMGR

Creates Default Queue Manager by Name MYQMGR Creates dead letter queue SYSYTEM.DEAD.LETTER.QUEUE

Strmqm Starts Queue Manager

runmqsc Define Local Queue

define qlocal(’QUEUE1’) replace descr (’test queue’)

Define Remote Queue DEFINE QREMOTE(Q1) RNAME(Q1) RQMNAME(QMB) XMITQ(QMB)


MQSeries Basics - API

1. MQCONN Connect to queue manager

2. MQDISC Disconnect from queue manager

3. MQOPEN Open queue

4. MQCLOSE Close queue

5. MQPUT Put message

6. MQGET Get message

7. MQPUT1 Put one message

8. MQBEGIN Begin unit of work

9. MQCMIT Commit

10. MQBACK Back out

11. MQINQ Inquire about queue attributes

12. MQSET Set queue attributes


MQSeries Basics - How MQ Series Works



The queuing system consists of the following parts: Queue Manager (MQM) Listener Trigger Monitor Channel Initiator Message Channel Agent (MCA) or mover

Application uses MQI to communicate with Queue Manager



Steps1. Application program issues MQPUT API call when it wants to put a

message on a queue.

2. Queue Manager checks if Queue is Local or Remote

3. If Local Queue QM adds relevant Message Headers and places message in the Queue

4. If Remote Queue QM adds relevant Message Headers and places the message in the XMIT Queue.

• Header contains information from the remote queue definition, such as destination queue manager name and destination queue name.

• Each remote queue must be associated with an xmit queue.

• Usually, all messages destined for one remote QueueManager use the same xmit queue



Steps…5. Transmission is done via channels.

i. Channels can be started automatically or manually

ii. To start a channel automatically• To start a channel automatically, the xmit queue must be associated with a channel

initiation queue, so that Queue Manager puts a init message to Channel initiation queue

• Channel initiator Queue is monitored by Channel Initiator• Channel Initiator starts the MCA for particular channel

iii. MCA moves the message over the network to the other machine, using the sender part of the message channel pair.

iv. Listener program monitors a specified port (default port is 1414)

v. On Message arrival at the port Listener start MCA at receiving end

vi. MCA moves the message into specified local queue using the receiver part of the message channel pair.

Note: Both channel definitions, sender and receiver, must have the same name.

6. Trigger Monitor processes the incoming message


MQSeries Basics – Communication


MQSeries Basics – Communication

Each machine has a queue manager installed Each queue manager manages several local queues Messages destined for a remote queue manager are put into a remote

queue. A remote queue is associated with a transmission (xmit) queue, which

is a local queue. Usually, there is one xmit queue for each remote queue manager.

A transmission queue is associated with a message channel. Message Channel transmits messages unidirectionally.


MQSeries Basics – Communication Components

Transmission Queue can be accessed in 2 ways:1. Remote Queue definitions

2. Direct Addressing in application• Queue Name

• Queue Manager Name

• No location transparency


MQSeries Basics – Two Way Communication Components

A remote queue definition that mirrors the local queue in the receiver machine and links to a transmission queue (Q1 in system A and Q2 in system B).

A transmission queue that holds all messages destined for the remote system until the channel transmits them (QMB in system A and QMA in system B).

A sender channel that gets messages from the xmit queue and transmits them to the other system using the existing network (QMA.QMB in system A and QMB.QM.A in system B).

A receiver channel that receives messages and puts them into a local queue (QMB.QMA in system A and QMA.QMB in system B); receiver channels can be started automatically by the queue manager when Channel Auto Definition (CHAD) is enabled.

A local queue from which the program gets its messages (Q2 in system A and Q1 in system B).


MQSeries Basics – Channel Concepts

Channels are unidirectional Two-way communication requires two channels Each Channel need to be defined twice

Once in the system that sends the message (sender channel) and once in the system that receives the message (receiver channel).

Each channel pair (sender and receiver) must have the same name.

MQSeries Cluster support has special channel types


MQSeries Basics – Communication Sample

System A (QMA) System B (QMB)

DEFINE QREMOTE(Q1) +RNAME(Q1) RQMNAME(QMB) +XMITQ(QMB) DEFINE QLOCAL(Q1)

DEFINE QLOCAL(QMB) +USAGE(xmitq)

DEFINE CHANNEL(QMA.QMB) +CHLTYPE(sdr) +XMITQ(QMB) +TRPTYPE(tcp) +CONNAME(9.24.104.123)

DEFINE CHANNEL(QMA.QMB) +CHLTYPE(rcvr) +TRPTYPE(tcp)

DEFINE QLOCAL(Q2)

DEFINE QREMOTE(Q2) +RNAME(Q2) RQMNAME(QMA) +XMITQ(QMA)

DEFINE QLOCAL(QMA) +

USAGE(xmitq)

DEFINE CHANNEL(QMB.QMA) +CHLTYPE(rcvr) +TRPTYPE(tcp)

DEFINE CHANNEL(QMB.QMA) +CHLTYPE(sdr) +XMITQ(QMA) +TRPTYPE(tcp) CONNAME(ABC1)



Start Communication Manually The commands to start listener and channel for queue manager QMA

are:strmqm QMAstart runmqlsr -t tcp -m QMA -p 1414runmqscstart channel (QMA.QMB)End

With the first command you start queue manager QMA, if not already started.

The next command starts the listener. Itlistens on behalf of QMA on port 1414. As transmission protocol TCP/IP is used.

The third command starts runmqsc in interactive mode. The channel QMA.QMB is started under control of runmqsc.

For the other queue manager you issue equivalent commands.



Start Communication Automatically Use the channel initiator to start channels.

start runmqlsr -t tcp -m QMA -p 1414 start runmqchi

• With the first command you start the listener

• With the second the channel initiator program.

For the other queue manager you issue equivalent commands.

Test the sample


MQ Series – Channel Initiator

Channel Initiator: A channel initiator acts as a trigger monitor for sender channels, because a

transmission queue may be defined as a triggered queue. When a message arrives on a transmission queue that satisfies the

triggering criteria for that queue, a message is sent to the initiation queue, triggering the channel initiator to start the appropriate sender channel.

You can also start server channels in this way if you specified the connection name of the partner in the channel definition.

This means that channels can be started automatically, based upon messages arriving on the appropriate transmission queue.


MQSeries Basics – Exits

An exit is a piece of code that is executed by the queue manager upon certain events. Those events include the placing of a message on a queue, the starting of a communications channel, the exchange of security information with a remote queue manager. Many exits are provided as add-on “SupportPacs”.


MQSeries Basics – Undelivered Messages

What happens to the undelivered Messages: Message-retry

• If the MCA is unable to put a message to the target queue for a reason that could be transitory (for example, because the queue is full), the MCA has the option to wait and retry the operation later.

• Specify a message-retry time and interval for MQPUT errors when you define your channel.

Return-to-sender• If message-retry was unsuccessful, or a different type of error was encountered,

the MCA can send the message back to the originator.

• Specify the following options in Message Descriptors• The MQRO_EXCEPTION_WITH_FULL_DATA report option

• The MQRO_DISCARD_MSG report option

• The name of the reply-to queue and reply-to queue manager

Dead-letter queue• If a message cannot be delivered or returned, it is put on to the dead-letter queue

(DLQ).


MQSeries Basics – Communication between Client and Server

Define a Client/Server Connection Clients and servers are connected with MQI channels. MQI channel consists of a sender/receiver pair, called Client Connection

(CLNTCONN) and Server Connection (SVCONN) channel Client connection channel is defined as an environment variable set MQSERVER=CHAN1/TCP/9.24.104.206(1414) where:

• MQSERVER is the name of the environment variable.

• CHAN1 is the name of the channel to be used for communication between client and server. This channel is defined in the server. MQSeries will automatically create it should it not exist.

• TCP denotes that TCP/IP is to be used to connect to the machine with the address following the parameter.

• 1414 is the default port number for MQSeries.

Definition of the server is as follows:

DEFINE CHANNEL(’CHAN1’) CHLTYPE(SVRCONN) REPLACE TRPTYPE(TCP) MCAUSER(’ ’)


MQSeries Basics – Triggering

Why Triggering? Some WebSphere MQ applications that serve queues run continuously, so

they are always available to retrieve messages that arrive on the queues. This may not be desirable when the number of messages arriving on the

queues is unpredictable. In this case, applications could be consuming system resources even when

there are no messages to retrieve.

What is MQ Triggering? WebSphere MQ provides a facility that enables an application to be started

automatically when there are messages available to retrieve. This facility is known as triggering.


MQSeries Basics – Triggering

Triggering allows Instantiation as required Conservation of system resources Automation of flow


MQSeries Basics – Triggering Components

Application queue Local queue, which, when it has triggering set on and when the conditions

are met, requires that trigger messages are written

Process Definition Holds details of the application that will get messages from the application

queue.

Transmission queue Required only when it is required to trigger a channel

Trigger Events Event that causes a trigger message to be generated by the queue

manager. Supported Triggering conditions are

• When the first message is put into the queue

• Every time a message is put into the queue

• When the queue contains a specified number of messages


MQSeries Basics – Triggering Components

Trigger message Message created by Queue Manager when it recognize an triggering event. Trigger messages have a fixed format Trigger messages contains information about

• Application to be started

• Application Queue

• Process Definition

Initiation queue Local queue on which the queue manager puts trigger messages. Queue manager can own more than one initiation queue

Trigger monitor Continuously-running program that serves one or more initiation queues When a trigger message arrives on an initiation queue, the trigger monitor retrieves

the message. The trigger monitor uses the information in the trigger message. It issues a command to start the application that is to retrieve the messages arriving

on the application queue, passing it information contained in the trigger message header, which includes the name of the application queue.


MQSeries Basics - Flow of application and trigger messages


MQSeries Basics - Flow of application and trigger messages

1. Application A, puts a message on the application queue for which triggering is enabled.

2. The queue manager checks to see if the conditions are met under which it has to generate a trigger event. They are, and a trigger event is generated. Information held within the associated process definition object is used when creating the trigger message.

3. The queue manager creates a trigger message and puts it on the initiation queue associated with this application queue, but only if an application (trigger monitor) has the initiation queue open for input.

4. The trigger monitor retrieves the trigger message from the initiation queue.

5. The trigger monitor issues a command to start application B

6. Application B opens the application queue and retrieves the message.


MQSeries Basics – Triggering Definitions

Create an initiation queue for your application queue. For example:DEFINE QLOCAL (initiation.queue) REPLACE +

LIKE (SYSTEM.DEFAULT.LOCAL.QUEUE) +

DESCR ('initiation queue description' )

The target queue must have “triggering” specified DEFINE QLOCAL (application.queue) REPLACE +

LIKE (SYSTEM.DEFAULT.LOCAL.QUEUE) +

DESCR ('appl queue description') +

INITQ ('initiation.queue') +

PROCESS ('process.name') +

TRIGGER +

TRIGTYPE (FIRST)

Process definition associated with the target queue can be this:DEFINE PROCESS(process.name) REPLACE +

DESCR(’Process to start server program’) +

APPLTYPE(WINDOWSNT) +

APPPLICID(‘c:\test\myprog.exe’)


MQSeries Basics – Trigger Types

EVERY

Every time a message is put in the target queue a trigger message is also put in the initiation queue. Use this when your program exits after processing one message or transaction, as shown above on the left.

FIRST

A trigger message is put in the initiation queue only when the target queue has been empty. Use this when the program exits only then when there are no more messages in the queue, as shown on the right.

n messages

A trigger message is put in the initiation queue when there are n messages in the target queue. For example, you can start a batch program when the queue holds 1000 messages.


MQ Series Clusters


MQ Cluster

Clusters are groups of machines connected together. Even multiple queue managers on a single processor can build or be

part of a cluster.

Why Clustering Process very high message rates.

Process very large numbers of messages.

Single system to control and administer.

System should always be available.


Cluster components

Cluster queue manager - Queue Manager that is a member of a cluster. Queue Managers can be part of multiple clusters (Overlapping clusters).

Cluster queue - Queue that is hosted by a cluster queue manager and made available to other queue managers in the cluster

Repository - A repository is a collection of information about the queue managers that are members of a cluster

Repository contains information such as queue manager names, their locations, their channels, which queues they host, etc.

Information is stored in form of messages in the a Queue called SYSTEM.CLUSTER.REPOSITORY.QUEUE

Typically, two queue managers in a cluster hold a full repository. The remaining queue managers all hold a partial repository.

Full repository - Queue manager that hosts a complete set of information about every queue manager in the cluster is referred to as having a full repository for the cluster.

Partial repository - Queue manager's partial repository contains information about only those queue managers with which the queue manager needs to exchange messages.


Cluster components

Cluster-receiver channel - similar to a receiver channel used in distributed queuing, but in addition to carrying messages this channel can also carry information about the cluster.

Cluster-sender channel - similar to a sender channel used in distributed queuing, but in addition to carrying messages this channel can also carry information about the cluster.

Cluster transmission queue - transmits all messages from the queue manager to any other queue manager that is in the same cluster.


MQ Clusters

Reasons for reduced MQSeries administration: Communication channels are created automatically. Queues are administered cluster-wide. No need to define explicit remote queues. Only one cluster transmission queue is required to send messages to

all queue managers in the cluster, and this queue is created automatically.


Distributed Queues & Clustering Comparison

Four Queue Managers, each with two local queues

Definitions for distributed queuing

Description Number per QM

Total number

A sender-channel definition for a channel on which to send messages to every other queue manager

3 12

A receiver-channel definition for a channel on which to receive messages from every other queue manager

3 12

A transmission-queue definition for a transmission queue to every other queue manager

3 12

A local-queue definition for each local queue 2 8

A remote-queue definition for each remote queue to which this queue manager wants to put messages

6 24

Optionally, on z/OS, a process definition specifying trigger data if channels are to be triggered

3 12

Definitions for clustering

DescriptionNumber per QM

Total number

A cluster-sender channel definition for a channel on which to send messages to a repository queue manager

1 4

A cluster-receiver channel definition for a channel on which to receive messages from other queue managers in the cluster

1 4

A local-queue definition for each local queue

2 8


Creating MQ Clusters

Default Objects associated with Clusters SYSTEM.CLUSTER.REPOSITORY.QUEUE

• This local queue is used to store all the repository information.

SYSTEM.CLUSTER.COMMAND.QUEUE• This local queue is used to carry messages to the repository.

SYSTEM.CLUSTER.TRANSMIT.QUEUE• This is the transmission queue for all messages to all queues and queue

managers that are within clusters.

SYSTEM.DEF.CLUSSDR• This is used to supply default values for any attributes that you do not specify

when you create a cluster sender channel on a queue manager in the cluster.

SYSTEM.DEF.CLUSRCVR• This is used to supply default values for any attributes that you do not specify

when you create a cluster-receiver channel on a queue manager in the cluster.


Creating MQ Clusters

Create a cluster with four queue managers using scripts

Create following Queue managers

QM Name TCP/IP Port

QM_1 1415

QM_2 1416

QM_3 1417

QM_4 1418

Create and Configure QM_1 & QM_3 first since they are full repositories.

mqcluster_demo.zip


MQSeries Basics – Samples & Exercises

Sample MQ Put

Sample MQ Get

MQ Browse

MQPut.java

MQGet.java

mqbrowse.java


Websphere MQ Series – Transaction Coordination


MQSeries Basics - Transaction Coordination

MQSeries is already an XA-compliant resource manager. Can participate in a two-phase commit coordinated by an XA transaction

manager Queue Manager is XA resource manager and an XA resource coordinator.


MQSeries Basics - Transaction Coordination - Basics

Commit and Backout: When a program puts a message on a queue within a unit of work, that

message is made visible to other programs only when the program commits the unit of work. To commit a unit of work, all updates must be successful to preserve data integrity. If the program detects an error and decides that the put operation should not be made permanent, it can back out the unit of work. When a program performs a back out, WebSphere MQ restores the queue by removing the messages that were put on the queue by that unit of work. The way in which the program performs the commit and back out operations depends on the environment in which the program is running.

Syncpoint coordination, syncpoint, unit of work Syncpoint coordination is the process by which units of work are either

committed or backed out with data integrity.



Unit of Work Global unit of work – Two Phase Commit

• Includes both database and queue updates

• Can update both queues and databases and need to be started with MQBEGIN.

• After the MQBEGIN call has been issued, local resources can be updated using MQGET, MQPUT or MQPUT1 calls made under syncpoint.

• The unit of work can be committed by an MQCMIT or an MQDISC call.

• Unit of work will be rolled back by an MQBACK call, or if the application terminates without first disconnecting.

Local unit of work - Single Phase Commit• Consists of queue updates only.

• Is started by an MQGET, MQPUT or MQPUT1 call which specifies the corresponding syncpoint option.

• Units of work can be committed explicitly by an MQCMIT call, or implicitly by an MQDISC call.

• Units of work can be rolled back explicitly by an MQBACK call, or implicitly if the application terminates without first disconnecting.

Global Unit of WorkMQCONN()….MQBEGIN()…..MQGET(MQGMO_SYNCPOINT)...EXEC SQL UPDATE...MQCMIT()MQDISC()

Local Unit of WorkMQCONN()...MQGET(MQGMO_SYNCPOINT)...MQCMIT()...MQPUT(MQPMO_SYNCPOINT)...MQDISC



Mixing Units of Work• Application can consist of both local and global units of work.• Units of work that make only queue manager updates can be started using an MQGET,

MQPUT or MQPUT1 call specifying the appropriate syncpoint option.

• Units of work that also need to update global resources owned by a database manager need to be started using an MQBEGIN call.

Valid Invalid Invalid

MQCONN ()

MQPUT (MQPMO_SYNCPOINT)

MQCMIT ()

......

MQBEGIN ()

MQGET (MQGMO_SYNCPOINT)

EXEC SQL UPDATE

MQCMIT ()

MQCONN ()

MQPUT (MQPMO_SYNCPOINT)

...

MQBEGIN ()

MQCONN ()

...

MQBEGIN ()

...

MQBEGIN ()

It is an error to try to start another unit of work while there is another already in progress.



Local Unit of Work Sample

mqsync.java


MQSeries Basics – Message Segmenting and Grouping

Definitions Physical Message

The basic unit that appears in a queue.

Logical Message A single unit of application information.

Message Segment Part of a logical message.


Websphere Message Segmenting and Grouping



Segmentation MQSeries (Queue Manager) or Application programmer can segment the

message Receiving application has the option to either receive the entire message in

one piece or each segment separately. Several physical messages build one logical message MQSeries (Queue Manager) ensures that the order of segments is

maintained. Used in situations where the Message length is more than that is allowed. Simplest way is to let MQSeries do the segmentation for you.

• The sending application has to inform the queue manager that it can segment the message if necessary, and the receiving application has to tell the queue manager to reassemble the message if it has been segmented.



To allow the queue manager to perform segmenting on its own when the message exceeds MAXMSGL, specify in the message descriptor:

• md.Version = MQMD_VERSION_2;

• md.MsgFlags = MQMF_SEGMENTATION_ALLOWED;

The receiving application can get each message segment individually or have the queue manager reassemble the message and return it to the program after all segments have been received.

The queue manager makes sure that the segments are in the correct order.• gmo.OPTIONS += MQGMO_COMPLETE_MSG;

Note: If not specified segments are returned individually.

Exercise: Arbitrary Segmentation Take a message which is too big (either for the queue manager or for the specific

queue) And automatically segment it

• MQSeries breaks the message into smaller messages (called segments) and sends them wherever the “too large” message would have been sent.

• The local queue manager at the receiving end puts the pieces back together and is able to present the “too large” message to the GETting application as though it had never been segmented.



Grouping Group several small logical

messages together and build one larger physical message

It is possible for a message that is segmented to be part of a group of messages.

Receiving application disassembles the message received.

Facilitates reduced network traffic. Messages in a group will all have

the same Group ID which can be controlled by the application or generated by the queue manager.

The logical sequence of messages in a group is maintained using the message sequence number field in the message header.


THANK YOU

MQSERIES Presentation

Documents

Transcript of MQSERIES Presentation