Rabbit MQ Connector

Introduction

The Neuron ESB Rabbit MQ Adapter allows Neuron ESB to interact with Rabbit MQ in a variety of ways, with configurable levels of Transaction support and Quality of Service. Organizations can use the adapter to monitor Rabbit MQ Topics or Queues for new messages, forwarding those (i.e. publishing) to a Neuron ESB Topic where other subscribing parties (e.g. Adapter, Service or Workflow Endpoints) can receive the information. Conversely, the adapter can also be used to forward information published to or generated by Neuron ESB to either a Rabbit MQ Queue or Topic, essentially making Rabbit MQ a Neuron ESB subscriber. Both types of Rabbit MQ Transactions are supported (e.g. Transaction and Publish Confirms) as are Virtual Hosts, SSL and all the latest features of Rabbit MQ.

Before reading this document

If you are not familiar with Neuron ESB Adapters, take a moment to consult our online help by clicking here. This document assumes you have the following knowledge:

1. What adapters are and how they are leveraged in Neuron ESB
2. Register a Neuron ESB Adapter within a Neuron ESB Solution
3. Create an Adapter Endpoint in a Neuron ESB Solution
4. Set a Neuron ESB Adapter Endpoint Properties
5. Invoke a Neuron ESB Adapter Endpoint

Prerequisites

The Neuron ESB Rabbit MQ Adapter supports version 3.5.6 and later of Rabbit MQ and version 18.1 or later of Erlang. Using the Neuron ESB installer, users can optionally select to install these during the setup process, or can download these from the locations list in our readme.html file or directly from the Rabbit MQ web site.

Supported Modes

The Rabbit MQ Adapter supports 2 modes:

• Publish: Used to monitor a Rabbit MQ Topic or Queue for new messages using Rabbit MQ’s Consumer Event based model. The information received from Rabbit MQ (as well as any custom headers) is published to a configured Neuron Topic.

• Subscribe: Used to send messages from Neuron ESB to a Rabbit MQ Topic or Queue. The adapter allows many of the properties controlling the destination of the message to be configured dynamically at runtime.

Rabbit MQ Adapter Properties

Every Neuron ESB Adapter has design time properties that can configured within a standard Microsoft property grid (as shown below). The Rabbit MQ Adapter properties are located on the Properties tab of the Adapter Endpoint by navigating to Connections->Endpoints->Adapter Endpoints within the Neuron Explorer. These properties are used to determine the behavior of the adapter at runtime. The Rabbit MQ Adapter Properties fall into four classes:

• General: Applies to modes (Publish and Subscribe)
• Publish Mode Properties: Applies only to the Publish Mode
• Security: Applies to the Connection made at runtime to Rabbit MQ regardless of mode.
• Subscribe Mode Properties: Applies only to the Subscribe Mode

Rabbit MQ Properties

Property	Category	Description
Connect on Startup	General	The controls whether or the adapter will attempt to connect to the Rabbit MQ server on startup. This only is relevant for Subscribe mode since Publish mode will immediately attempt a connection on startup.
Connection Timeout	General	Timeout setting for connection attempts (in seconds).
Machine Name	General	The name or IP address of the Rabbit MQ Server to connect to.
Port	General	The port number of the Rabbit MQ Server. -1 uses default port
Username	General	User Id for Rabbit MQ Server logon
Password	General	The password for Rabbit MQ Server logon.
Virtual Host	General	Virtual Host (e.g vHost) of Rabbit MQ Server. Default is “/”
Protocol	General	The Rabbit MQ Protocol to use. Will use default from Rabbit MQ server app settings if not provided. Rabbit MQ AMQP 0.9.1 is supported
Publish Topic	Publish	This property is used to specify the Neuron ESB Topic to use when publishing the received message to the bus.
Auto Acknowledge	Publish	If true, messages will be automatically acknowledged and removed from the queue once the party receives the message.
Prefetch Size	Publish	The number of messages that will be pre-fetched from the queue to transport layer. If 'Auto Acknowledge' is set to true, there will be a possibility of message loss if the Rabbit MQ server fails. A value of 0 means unlimited.
Subscription Type	Publish	Determines what and how to monitor on Rabbit MQ to receive messages. Queue = Receives messages from an existing Queue. Topic = Receives messages from a Queue where messages are from a specific Exchange with a Specific Routing key.
-Queue	Publish	Rabbit MQ Queue to receive messages from
-Exchange	Publish	If Subscription Type = Topic. Rabbit MQ Exchange to receive messages from. This is used to create a Subscription based on Exchange, Queue and Routing key.
-Subscription	Publish	If Subscription Type = Topic. Routing Key used to receive messages from. Acts as a subscription. Example : 'STOCK.IBM.#', Where '*' can be used to replace one word and '#' to replace 0 or more words
-Persist Subscription	Publish	If Subscription Type = Topic. If True, the Subscription will remain after the adapter is shut down. If false, the Subscription will be removed when adapter shuts down. The Subscription is essentially the Routing Key that the Queue is bound to.
Error Reporting	Publish	This property determines how all errors are reported in Event Log and Neuron Logs. Either as Errors, Warnings or Informational. This property can be one of three values: ·       Error (Default) ·       Information ·       Warning
Error on Publish	Publish	This property determines if monitoring of the Rabbit MQ Server continues on error and if consecutive errors are reported. The property is used to determine consecutive reporting and whether or not it should shut down on if an exception occurs in the event handler. This property can be one of three values: ·       StopPollingOnError (Default): Stops the Adapter when an error is encountered. ·       SuppressConsecutiveErrors: The Adapter will only report the first error and will continue to run. ·       ReportAllErrors: The Adapter will only report all errors and will continue to run.
SSL Enabled	Security	Connect to Rabbit MQ Server using only SSL
-Port	Security	SSL Port for all Rabbit MQ server connections
-SSL Protocol	Security	SSL Protocol to use for all Rabbit MQ server connections.
-Client Authentication	Security	Require Neuron ESB to provide Rabbit MQ Server a client certificate to authenticate against.
-Certificate	Security	Select a client certificate configured in the Security section of the Neuron ESB Explorer to authenticate against the Rabbit MQ Server.
-Passphrase	Security	Passphrase for the client certificate, if one exists
Delivery Mode	Subscribe	Persistent (i.e. Durable) or NonPersistent. Persistence ensures message is written to disk. If using Transactions, Persistence SHOULD be used.
Routing Mode	Subscribe	Determines if the message should be sent to a Queue or to a Topic (i.e. Exchange with Routing Key). Queues that are bound to the Routing Key and Exchange will receive the message. Routing Keys can be used to form Topic based subscriptions. Example : 'STOCK.IBM.#', Where '*' can be used to replace one word and '#' to replace 0 or more words
-Queue	Subscribe	If Routing Mode = Queue. Rabbit MQ Queue to send messages to
-Exchange	Subscribe	If Routing Mode = Topic. Rabbit MQ Exchange to send messages to. Used in conjunction with Routing key
-Topic	Subscribe	If Routing Mode = Topic. Routing Key used to send messages. Acts as a Topic. Example : 'STOCK.IBM.#', Where '*' can be used to replace one word and '#' to replace 0 or more words
Transaction Type	Subscribe	Controls the level of reliability for messages. Either 'None', 'PublisherConfirms' or 'Transactional' can be selected. 'PublisherConfirms' uses an asynchronous Ack/Nack protocol while 'Transactional' forces a commit/rollback on each message published.
-Batch Size	Subscribe	Only for use with PublishConfirms type of transactions. The number of messages that will be published to Rabbit MQ in a Publish Confirm transaction
-Batch Confirm Timeout	Subscribe	Only for use with PublishConfirms type of transactions. The number of seconds to wait after the Batch of messages have been published to receive all ACKs/NACKs from Rabbit MQ. Should be a value between 1 and 60.
-Inactivity Timeout	Subscribe	Only for use with PublishConfirms type of transactions. The number of minutes to wait after the last message sent before checking to determine if all ACKs/NACKs from Rabbit MQ have been received. Should be a value between 1 and 5.
Time To Live	Subscribe	A value in minutes that specifies how long messages are valid for delivery by Rabbit MQ before they are expired (dead letter).
Must be Routable	Subscribe	If set to true, the message must be routable by Rabbit MQ. If the message cannot be routed to a destination queue, the failure will be recorded asynchronously and posted to the Neuron ESB Failure Message database table.

MetaData

Neuron ESB has the ability to support custom message properties, either generated by users or by the sub systems or Adapters that process the message. In messaging systems, it’s very common to add custom meta data to a message where its either impractical or impossible to alter the original inbound or outbound message. Many developers use these custom properties to assist in driving custom business logic as part of the integration process.

For example, when the Rabbit MQ Adapter receives a message from a Queue or Topic, that information is stored in the Neuron ESB Message Body property. However, the adapter adds a number of custom properties specific to the adapter to the Neuron ESB Message before its published. These properties would contain information such as the Name of the Queue or Topic the message was retrieved from as well as any Rabbit MQ custom headers which may have already existed on the message. The following shows a message retrieved by the Rabbit MQ adapter, published to a Neuron ESB Topic and received by a subscribing Neuron ESB Test Client.

All custom message properties generated by the Rabbit MQ adapter are all prefixed with the moniker “rmq”. These properties can be retrieved or set within any Business Process or Workflow Activity using the Set Property Process Step or, in any C# Editor, using the following:

var routingKey = context.Data.GetProperty("rmq","RoutingKey");

var customHeader = context.Data.GetProperty("rmq","Header.custom1");

If users do NOT want Neuron ESB to either generate or set custom message properties specific to the Rabbit MQ adapter (i.e. properties prefixed with “rmq”), the “Include Metadata Properties” located on the General tab of the Adapter Endpoint should be unchecked.

Publish Mode

The following properties can be generated and added to the Neuron ESB Message’s Custom Properties collection using the moniker/prefix, “rmq”.

Property	Source	Description
Mode	Neuron ESB	Indicative of the Mode the Rabbit MQ Adapter is configured for. Generated by Neuron ESB
Server	Neuron ESB	Rabbit MQ server. Generated by Neuron ESB
Port	Neuron ESB	Rabbit MQ server port. Generated by Neuron ESB
Queue	Neuron ESB	Rabbit MQ Queue the message was received from. Generated by Neuron ESB
ConsumerTag	Rabbit MQ	Read from Rabbit MQ Basic Properties
DeliveryTag	Rabbit MQ	Read from Rabbit MQ Basic Properties
RoutingKey	Rabbit MQ	Read from Rabbit MQ Basic Properties
DeliveryMode	Rabbit MQ	Read from Rabbit MQ Basic Properties
Persistent	Rabbit MQ	Read from Rabbit MQ Basic Properties
Expiration	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
ContentType	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
ClusterId	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
ContentEncoding	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
CorrelationId	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
AppId	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
MessageId	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
Priority	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
ReplyTo	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
UserId	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties
ReplyToAddress.ExchangeName	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present.
ReplyToAddress.ExchangeType	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present
ReplyToAddress.RoutingKey	Rabbit MQ	Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present
Exchange	Neuron ESB	Rabbit MQ Exchange the message on the Queue was received from. If Queue, generated by Rabbit MQ, if Topic was configured, generated by Neuron ESB.
Subscription	Neuron ESB	Rabbit MQ Subscription that was used to retrieve the message. Only if message is retrieved via Topic. Generated by Neuron ESB.
Header.*	Rabbit MQ	Optional. Read from the Rabbit MQ Basic Properties Headers collection. * represents the name of the header retrieved.

Subscribe Mode

Dynamic Configuration Properties

When the Rabbit MQ Adapter is configured in Subscribe mode, some of the adapter’s specific custom message properties can be used to dynamically configure the Adapter Endpoint at runtime. For example, at runtime a Business Process or Workflow may execute business logic that determines what Queue, Exchange or Routing Key should be used to for the Adapter Endpoint. The properties that can be set at runtime are:

• Exchange
• Queue
• RoutingKey
• Expiration

Setting the properties can be done using the Set Properties Process Step or by using C# code in a Neuron ESB Code Editor as shown below:

context.Data.SetProperty("rmq","RoutingKey","TestTopic");

Add Custom Headers

Adding extra meta data to the message that Neuron ESB sends to a Rabbit MQ Queue or Exchange can be easily done using the “Header.*” type properties of the adapter. Recipients using Rabbit MQ can retrieve theses using the Rabbit MQ API and accessing the Headers collection of the Basic Properties of the Rabbit MQ message. This meta data can be written at runtime by using C# code in a Neuron ESB Code Editor as shown below:

context.Data.SetProperty("rmq","Header.custom1","my extra info");

Publish Confirm Transactions

The Neuron ESB Rabbit MQ Adapter supports both Transaction types that Rabbit MQ offers; their channel based Transaction model as well as their batched style Transaction model e.g. Publish Confirms. Both are Acknowledge, Negative Acknowledgement (ack/nack) based models. Users can learn more about Publish Confirms here: https://www.rabbitmq.com/confirms.html , as well as why Rabbit MQ introduced them: https://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms .

The Rabbit MQ Adapter exposes several properties that can be used to finely tune the performance and throughput as well the reliability of the batch transaction such as “Batch Confirm Timeout” and “Inactivity Timeout”. These properties force Neuron ESB to call into Rabbit MQ for it to finish sending any pending acks/nacks. Messages the adapter receives nacks for or where we’re notified by Rabbit MQ that that message is undeliverable (i.e. where “Must be Routable” property is set to True), are automatically moved into the Neuron ESB Failed database table with an AdapterSubscribeException type.

Security

With CU4’s support for Rabbit MQ 3.5.6 comes support for SSL. Once SSL has been enabled, the Rabbit MQ SSL port needs to be provided and the SSL Protocol to use must be selected. Although Neuron ESB and Rabbit MQ supports both SSL2, SSL3, TLS, TLS 1.1 and TLS 1.2 by default SSL3 support is disabled by Rabbit MQ to avoid POODLE attacks. More about Rabbit MQ and its SSL support can be found here: https://www.rabbitmq.com/ssl.html

Client Authentication can also be enabled by providing a Certificate (registered within the Security section of the Neuron ESB Explorer).

Configuring SSL support is done by modifying the Rabbit MQ configuration file as well as registering several important Environment Variables for the machine. More information can be found here: https://www.rabbitmq.com/configure.html .