Rabbit MQ Adapter
Introduction
The Neuron ESB Rabbit MQ Adapter allows Neuron ESB to interact with Rabbit MQ in a variety of ways, with configurable levels of Transaction support and Quality of Service. Organizations can use the adapter to monitor Rabbit MQ Topics or Queues for new messages, forwarding those (i.e. publishing) to a Neuron ESB Topic where other subscribing parties (e.g. Adapter, Service or Workflow Endpoints) can receive the information. Conversely, the adapter can also be used to forward information published to or generated by Neuron ESB to either a Rabbit MQ Queue or Topic, essentially making Rabbit MQ a Neuron ESB subscriber. Both types of Rabbit MQ Transactions are supported (e.g. Transaction and Publish Confirms) as are Virtual Hosts, SSL and all the latest features of Rabbit MQ.
Before reading this document
If you are not familiar with Neuron ESB Adapters, take a moment to consult the Adapter Overview. This document assumes you have the following knowledge:
- What adapters are and how they are leveraged in Neuron ESB
- Register a Neuron ESB Adapter within a Neuron ESB Solution
- Create an Adapter Endpoint in a Neuron ESB Solution
- Set a Neuron ESB Adapter Endpoint Properties
- Invoke a Neuron ESB Adapter Endpoint
Prerequisites
The Neuron ESB Rabbit MQ Adapter supports version 3.5.6 and later of Rabbit MQ and version 18.1 or later of Erlang. Using the Neuron ESB installer, users can optionally select to install these during the setup process, or can download these from the locations list in our readme.html file or directly from the Rabbit MQ web site.
Supported Modes
The Rabbit MQ Adapter supports 2 modes:
- Publish: Used to monitor a Rabbit MQ Topic or Queue for new messages using Rabbit MQs Consumer Event based model. The information received from Rabbit MQ (as well as any custom headers) is published to a configured Neuron Topic.
- Subscribe: Used to send messages from Neuron ESB to a Rabbit MQ Topic or Queue. The adapter allows many of the properties controlling the destination of the message to be configured dynamically at runtime.
Rabbit MQ Adapter Properties
Every Neuron ESB Adapter has design time properties that can configured within a standard Microsoft property grid (as shown below). The Rabbit MQ Adapter properties are located on the Properties tab of the Adapter Endpoint by navigating to Connections->Endpoints->Adapter Endpoints within the Neuron Explorer. These properties are used to determine the behavior of the adapter at runtime. The Rabbit MQ Adapter Properties fall into four classes:
- General: Applies to modes (Publish and Subscribe)
- Publish Mode Properties: Applies only to the Publish Mode
- Security: Applies to the Connection made at runtime to Rabbit MQ regardless of mode.
- Subscribe Mode Properties: Applies only to the Subscribe Mode
Rabbit MQ Properties
| Property | Category | Description |
| Connect on Startup | General | The controls whether or the adapter will attempt to connect to the Rabbit MQ server on startup. This only is relevant for Subscribe mode since Publish mode will immediately attempt a connection on startup. |
| Connection Timeout | General | Timeout setting for connection attempts (in seconds). |
| Machine Name | General | The name or IP address of the Rabbit MQ Server to connect to. |
| Port | General | The port number of the Rabbit MQ Server. -1 uses default port |
| Username | General | User Id for Rabbit MQ Server logon |
| Password | General | The password for Rabbit MQ Server logon. |
| Virtual Host | General | Virtual Host (e.g vHost) of Rabbit MQ Server. Default is / |
| Protocol | General | The Rabbit MQ Protocol to use. Will use default from Rabbit MQ server app settings if not provided. Rabbit MQ AMQP 0.9.1 is supported |
| Publish Topic | Publish | This property is used to specify the Neuron ESB Topic to use when publishing the received message to the bus. |
| Auto Acknowledge | Publish | If true, messages will be automatically acknowledged and removed from the queue once the party receives the message. |
| Prefetch Size | Publish | The number of messages that will be pre-fetched from the queue to transport layer. If ‘Auto Acknowledge’ is set to true, there will be a possibility of message loss if the Rabbit MQ server fails. A value of 0 means unlimited. |
| Subscription Type | Publish | Determines what and how to monitor on Rabbit MQ to receive messages. Queue = Receives messages from an existing Queue. Topic = Receives messages from a Queue where messages are from a specific Exchange with a Specific Routing key. |
| -Queue | Publish | Rabbit MQ Queue to receive messages from |
| -Exchange | Publish | If Subscription Type = Topic. Rabbit MQ Exchange to receive messages from. This is used to create a Subscription based on Exchange, Queue and Routing key. |
| –Subscription | Publish | If Subscription Type = Topic. Routing Key used to receive messages from. Acts as a subscription. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words |
| –Persist Subscription | Publish | If Subscription Type = Topic. If True, the Subscription will remain after the adapter is shut down. If false, the Subscription will be removed when adapter shuts down. The Subscription is essentially the Routing Key that the Queue is bound to. |
| Error Reporting | Publish | This property determines how all errors are reported in Event Log and Neuron Logs. Either as Errors, Warnings or Informational. This property can be one of three values:Error (Default)InformationWarning |
| Error on Publish | Publish | This property determines if monitoring of the Rabbit MQ Server continues on error and if consecutive errors are reported. The property is used to determine consecutive reporting and whether or not it should shut down on if an exception occurs in the event handler. This property can be one of three values:StopPollingOnError (Default): Stops the Adapter when an error is encountered.SuppressConsecutiveErrors: The Adapter will only report the first error and will continue to run.ReportAllErrors: The Adapter will only report all errors and will continue to run. |
| SSL Enabled | Security | Connect to Rabbit MQ Server using only SSL |
| -Port | Security | SSL Port for all Rabbit MQ server connections |
| -SSL Protocol | Security | SSL Protocol to use for all Rabbit MQ server connections. |
| -Client Authentication | Security | Require Neuron ESB to provide Rabbit MQ Server a client certificate to authenticate against. |
| -Certificate | Security | Select a client certificate configured in the Security section of the Neuron ESB Explorer to authenticate against the Rabbit MQ Server. |
| -Passphrase | Security | Passphrase for the client certificate, if one exists |
| Delivery Mode | Subscribe | Persistent (i.e. Durable) or NonPersistent. Persistence ensures message is written to disk. If using Transactions, Persistence SHOULD be used. |
| Routing Mode | Subscribe | Determines if the message should be sent to a Queue or to a Topic (i.e. Exchange with Routing Key). Queues that are bound to the Routing Key and Exchange will receive the message. Routing Keys can be used to form Topic based subscriptions. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words |
| –Queue | Subscribe | If Routing Mode = Queue. Rabbit MQ Queue to send messages to |
| –Exchange | Subscribe | If Routing Mode = Topic. Rabbit MQ Exchange to send messages to. Used in conjunction with Routing key |
| –Topic | Subscribe | If Routing Mode = Topic. Routing Key used to send messages. Acts as a Topic. Example : ‘STOCK.IBM.#’, Where ‘*’ can be used to replace one word and ‘#’ to replace 0 or more words |
| Transaction Type | Subscribe | Controls the level of reliability for messages. Either ‘None’, ‘PublisherConfirms’ or ‘Transactional’ can be selected. ‘PublisherConfirms’ uses an asynchronous Ack/Nack protocol while ‘Transactional’ forces a commit/rollback on each message published. |
| -Batch Size | Subscribe | Only for use with PublishConfirms type of transactions. The number of messages that will be published to Rabbit MQ in a Publish Confirm transaction |
| -Batch Confirm Timeout | Subscribe | Only for use with PublishConfirms type of transactions. The number of seconds to wait after the Batch of messages have been published to receive all ACKs/NACKs from Rabbit MQ. Should be a value between 1 and 60. |
| -Inactivity Timeout | Subscribe | Only for use with PublishConfirms type of transactions. The number of minutes to wait after the last message sent before checking to determine if all ACKs/NACKs from Rabbit MQ have been received. Should be a value between 1 and 5. |
| Time To Live | Subscribe | A value in minutes that specifies how long messages are valid for delivery by Rabbit MQ before they are expired (dead letter). |
| Must be Routable | Subscribe | If set to true, the message must be routable by Rabbit MQ. If the message cannot be routed to a destination queue, the failure will be recorded asynchronously and posted to the Neuron ESB Failure Message database table. |
MetaData
Neuron ESB has the ability to support custom message properties, either generated by users or by the sub systems or Adapters that process the message. In messaging systems, its very common to add custom meta data to a message where its either impractical or impossible to alter the original inbound or outbound message. Many developers use these custom properties to assist in driving custom business logic as part of the integration process.
For example, when the Rabbit MQ Adapter receives a message from a Queue or Topic, that information is stored in the Neuron ESB Message Body property. However, the adapter adds a number of custom properties specific to the adapter to the Neuron ESB Message before its published. These properties would contain information such as the Name of the Queue or Topic the message was retrieved from as well as any Rabbit MQ custom headers which may have already existed on the message. The following shows a message retrieved by the Rabbit MQ adapter, published to a Neuron ESB Topic and received by a subscribing Neuron ESB Test Client.
All custom message properties generated by the Rabbit MQ adapter are all prefixed with the moniker rmq. These properties can be retrieved or set within any Business Process or Workflow Activity using the Set Property Process Step or, in any C# Editor, using the following:
var routingKey = context.Data.GetProperty("rmq","RoutingKey");
var customHeader = context.Data.GetProperty("rmq","Header.custom1");
If users do NOT want Neuron ESB to either generate or set custom message properties specific to the Rabbit MQ adapter (i.e. properties prefixed with rmq), the Include Metadata Properties located on the General tab of the Adapter Endpoint should be unchecked.
Publish Mode
The following properties can be generated and added to the Neuron ESB Messages Custom Properties collection using the moniker/prefix, rmq.
| Property | Source | Description |
| Mode | Neuron ESB | Indicative of the Mode the Rabbit MQ Adapter is configured for. Generated by Neuron ESB |
| Server | Neuron ESB | Rabbit MQ server. Generated by Neuron ESB |
| Port | Neuron ESB | Rabbit MQ server port. Generated by Neuron ESB |
| Queue | Neuron ESB | Rabbit MQ Queue the message was received from. Generated by Neuron ESB |
| ConsumerTag | Rabbit MQ | Read from Rabbit MQ Basic Properties |
| DeliveryTag | Rabbit MQ | Read from Rabbit MQ Basic Properties |
| RoutingKey | Rabbit MQ | Read from Rabbit MQ Basic Properties |
| DeliveryMode | Rabbit MQ | Read from Rabbit MQ Basic Properties |
| Persistent | Rabbit MQ | Read from Rabbit MQ Basic Properties |
| Expiration | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| ContentType | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| ClusterId | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| ContentEncoding | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| CorrelationId | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| AppId | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| MessageId | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| Priority | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| ReplyTo | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| UserId | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties |
| ReplyToAddress.ExchangeName | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present. |
| ReplyToAddress.ExchangeType | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present |
| ReplyToAddress.RoutingKey | Rabbit MQ | Optional. Read from Rabbit MQ Basic Properties ONLY if ReplyToAddress is present |
| Exchange | Neuron ESB | Rabbit MQ Exchange the message on the Queue was received from. If Queue, generated by Rabbit MQ, if Topic was configured, generated by Neuron ESB. |
| Subscription | Neuron ESB | Rabbit MQ Subscription that was used to retrieve the message. Only if message is retrieved via Topic. Generated by Neuron ESB. |
| Header.* | Rabbit MQ | Optional. Read from the Rabbit MQ Basic Properties Headers collection. * represents the name of the header retrieved. |
Subscribe Mode
Dynamic Configuration Properties
When the Rabbit MQ Adapter is configured in Subscribe mode, some of the adapters specific custom message properties can be used to dynamically configure the Adapter Endpoint at runtime. For example, at runtime a Business Process or Workflow may execute business logic that determines what Queue, Exchange or Routing Key should be used to for the Adapter Endpoint. The properties that can be set at runtime are:
- Exchange
- Queue
- RoutingKey
- Expiration
Setting the properties can be done using the Set Properties Process Step or by using C# code in a Neuron ESB Code Editor as shown below:
context.Data.SetProperty("rmq","RoutingKey","TestTopic");
Add Custom Headers
Adding extra meta data to the message that Neuron ESB sends to a Rabbit MQ Queue or Exchange can be easily done using the Header.* type properties of the adapter. Recipients using Rabbit MQ can retrieve theses using the Rabbit MQ API and accessing the Headers collection of the Basic Properties of the Rabbit MQ message. This meta data can be written at runtime by using C# code in a Neuron ESB Code Editor as shown below:
context.Data.SetProperty("rmq","Header.custom1","my extra info");
Publish Confirm Transactions
The Neuron ESB Rabbit MQ Adapter supports both Transaction types that Rabbit MQ offers; their channel based Transaction model as well as their batched style Transaction model e.g. Publish Confirms. Both are Acknowledge, Negative Acknowledgement (ack/nack) based models. Users can learn more about Publish Confirms here: https://www.rabbitmq.com/confirms.html , as well as why Rabbit MQ introduced them: https://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms .
The Rabbit MQ Adapter exposes several properties that can be used to finely tune the performance and throughput as well the reliability of the batch transaction such as Batch Confirm Timeout and Inactivity Timeout. These properties force Neuron ESB to call into Rabbit MQ for it to finish sending any pending acks/nacks. Messages the adapter receives nacks for or where were notified by Rabbit MQ that that message is undeliverable (i.e. where Must be Routable property is set to True), are automatically moved into the Neuron ESB Failed database table with an AdapterSubscribeException type.
Security
With CU4s support for Rabbit MQ 3.5.6 comes support for SSL. Once SSL has been enabled, the Rabbit MQ SSL port needs to be provided and the SSL Protocol to use must be selected. Although Neuron ESB and Rabbit MQ supports both SSL2, SSL3, TLS, TLS 1.1 and TLS 1.2 by default SSL3 support is disabled by Rabbit MQ to avoid POODLE attacks. More about Rabbit MQ and its SSL support can be found here: https://www.rabbitmq.com/ssl.html.
Client Authentication can also be enabled by providing a Certificate (registered within the Security section of the Neuron ESB Explorer).
Configuring SSL support is done by modifying the Rabbit MQ configuration file as well as registering several important Environment Variables for the machine. More information can be found here: https://www.rabbitmq.com/configure.html.