FAQ: Anypoint MQ
This document lists frequently asked questions about Anypoint MQ.
Anypoint MQ requires a separate Enterprise subscription and is not included in the Anypoint Platform trial version. Contact MuleSoft Support for more information. If you don’t have an Enterprise subscription, you can use the MuleSoft Active MQ or Rabbit MQ.
See the Anypoint MQ starting page for the regions MQ and FIFO are available in.
If the blue create button doesn’t list FIFO Queue:
Ensure you have an Anypoint MQ FIFO entitlement. Check with your MuleSoft representative.
Ensure that your region setting is correct — see the Anypoint MQ starting page for the regions FIFO is available in.
After you obtain an Enterprise subscription and contacted your MuleSoft representative to activate Anypoint MQ, you can use Anypoint Platform to create a queue or message exchange, and send messages to it. For more information, see the MQ Tutorial.
MQ messages are always transported using in-flight HTTPS encryption with the latest Transport Layer Security (TLS) protocols. Additionally, if your Anypoint MQ queues are configured for encryption at rest, then the payloads are encrypted server-side with password-based encryption using PBE with MD5 and Triple DES, and with a 168-bit key.
If your application requires an additional layer of encryption security, then you can encrypt the payloads yourself before publishing messages to Anypoint MQ, but in this scenario you must manage your own encryption keys for both the message publishers and the consumers.
When you have a queue encrypted, the messages are stored encrypted but they are decrypted when they are read - this is automatic and transparent to the final user. There’s no option to disable the automatic decryption. If you need to encrypt the message so that the payload remains encrypted, manually encrypt the content.
No, queues and message exchanges are unique to the region in which they were created and cannot share messages or queues between regions. Developers can manually create custom programs that load balance between regions, but Anypoint MQ itself does not provide multi-region support.
Queues in each region are separate from those in other regions. You can name queues the same in each region, but queues can’t share messages across regions.
In Anypoint Platform > MQ > Message Browser you can fetch up to 50 non-FIFO messages. Up to 10 FIFO messages can be fetched. The FIFO restriction is because there can only be up to 10 in flight FIFO messages. For more information on in flight messages, see How many in flight messages can I have per queue?
Anypoint MQ is a highly distributed messaging system, so the results of any one API call to receive messages are not guaranteed to be entirely representative of what is fully in the queue. Anypoint MQ operates in a best-effort case. For example, if there are 5 messages in a queue, the system returns anywhere from 0-5 messages in a single API call depending on what distributed subsystems are accessed by that request. Subsequent API calls retrieve more messages and if you keep making API requests to receive messages, then you get all of them. Normally in high throughput cases this happens extremely fast. The MQ connector for example continuously polls the MQ backend for new messages and from the user perspective the messages process nearly instantly even though under the hood that could have resulted in many API calls.
Anypoint MQ limits messages to 10 MB. If a larger size is required, you can split the payload into multiple messages, or implement a claim check style service where you store the payload in a file system or blob storage, and then put a pointer to the payload in the message so it can be downloaded later. However, you need to manage access control to the blob storage for both the sender and receiver. You also need to consider that messages can be received unordered unless you use a FIFO queue.
Are message exchanges and queues completely unique and only accessible in their created environment?
That is, how do we ensure message exchanges or queues created in one environment are not confused or connected to message exchanges or queues created in a different environment if they have the same name?
You can have the same object name for queues and exchanges between environments, but the Client ID and Client Secret must be different. It’s not possible to access a production queue from a development environment if they don’t have the corresponding application client IDs.
You can use Anypoint Platform > Access Management > Business Groups or Environments. Queues and message exchanges created in a business group or environment are only visible to those with access to the business group or environment.
You can publish a message from an on-premises system to Anypoint MQ and have another on-premises subscriber pull messages off of Anypoint MQ. Currently Anypoint MQ cannot be deployed on-premises. If you need a messaging source on-premises, you can use MuleSoft Active MQ instead.
Within any given region, MQ services are deployed to multiple availability zones (AZs) to provide high availability (HA). If service in one availability zone goes down, the Anypoint MQ service still operates normally in that region. If all of the availability zones go down in a region, then the MQ service is unavailable in that region until at least one of the availability zones comes back up. However, the storage solution for MQ is durable, which means that there is no message loss for messages that were already in the MQ system before service was interrupted.
Anypoint MQ does not have built-in support for duplicating messages or queues across regions.
The Anypoint MQ connector does not exactly use a connection-based protocol, but uses REST behind the scenes, and therefore, you cannot use reconnection strategies with this connector.
On the inbound side, you can easily mimic a retry strategy using a max redelivery attribute set to your maximum number of retries and an exception strategy to move to a DLQ when the limit is hit.
On the outbound side, stick to the same triggering mechanism. Otherwise you can use the until-successful element with this connector.
You should also configure the HTTP connector so that Global Element Properties > Set Max Redelivery is set to the
There is no built-in functionality to replay messages. Thus, you need to implement a special flow or application to consume those messages and forward them back to original queue for reprocessing.
The network that Anypoint MQ runs on provides high availability replications across its many datacenters.
If a server fails and failover occurs, messages continue to be processed on other servers in the network on which Anypoint MQ runs. Normal Anypoint MQ queues do not guarantee only-once message delivery, only FIFO queues support only-once message delivery. The high availability network deduplicates messages for FIFO queues automatically.
Yes, Anypoint MQ guarantees "at least once" delivery of messages to the destination.
Every API call to Anypoint MQ counts as one API request. This includes sending, receiving, and acknowledging messages as well all operations on queues and exchanges.
Up to 10 messages can be retrieved from a single API call, which only bills as one request. Even if the request to retrieve messages does not return a message, like if the queue is empty, it still bills as a single API request.
There is no maximum transactions per second (TPS) for normal queues or exchanges. FIFO queues have a limit of 300 TPS; however, if you batch 10 messages per each read and write operation (maximum) using the API, FIFO queues can support up to 3,000 TPS.
Anypoint MQ supports up to 120,000 in flight messages per each non-FIFO queue. FIFO queues permit up to 10 in flight messages per FIFO queue due to the extra processing required for FIFO queues. An in flight message is a message received by a queue, but not deleted, that is, a message awaiting ACK or NACK, or a message with an expired Default Lock TTL (time-to-live) setting. A single queue can contain an unlimited number of messages; however the number of in flight messages is limited. The maximum duration for any message, either in flight or not is 2 weeks, after which Anypoint MQ deletes the message.
The connector does not store the message for resending it. The connector retries 5 times after which the message is discarded and it becomes the responsibility of the app for what to do with a message.
Retries for sending messages to Anypoint MQ broker are always synchronous. By contrast, the client mode specifies how to establish the connection to the backend and does does not govern retries of message sending.
Retries are arbitrary, maxRedelivery refers to a parameter which comes with the message saying how many times the messages were delivered but not processed (either NACK or TimeOut).
The MQ connector can process at most 10 messages in a queue, but that’s related to the prefetch configuration. The connector does not queue, if fetches at most 10 messages and processes them. The connector does not have an internal queue for later processing messages.
Yes, the REST API supports the
batchSize query parameter which lets you retrieve up to 10 messages in a single call (default value). The maximum number that can be retrieved are 10 messages in a single call, you can configure a lower value with the Prefetch Config parameter in the Anypoint MQ Connector. Note the number of messages retrieved by the connector can be less that the amount configured. A number higher that 10 can be configured but it is overriden.
If you have non-Mule applications, you can use our MQ REST API to send and receive messages.
Anypoint MQ is horizontally scalable and supports higher throughputs as needed.
Anypoint MQ provides long polling. You can do a REST request and ask the server to keep the TCP socket open for up to 20 seconds to fulfill your request if there are not enough messages.
To process messages one by one, set the Anypoint MQ connector to the consume operation, which retrieves a message from the queue, or receives null if a message is not available. In addition, use a synchronous flow with a poll scope to fire the process regularly. More than one in-flight message can occur if the process time between
anypoint-mq:ack is not lower than the Default Lock TTL for the queue, and if you don’t create exception options and NACK the message accordingly.
1 2 3 4 5 6 7 8 9 <flow name="testanypointmq1by1Flow2" processingstrategy="synchronous"> <poll doc:name="Poll"> <logger doc:name="Logger" level="INFO" message="Pooling fired"></logger> </poll> <anypoint-mq:consume config-ref="Anypoint_MQ_Configuration" destination="queuename" doc:name="Anypoint MQ"/> <logger doc:name="Logger" level="INFO" message="Processing message received. #[payload]"/> <anypoint-mq:ack config-ref="Anypoint_MQ_Configuration" doc:name="Anypoint MQ"/> <logger doc:name="Logger" level="INFO" message="Message processed."/> </flow>
Anypoint MQ provides direct access to the message ID and payload. You can see the message headers using the Chrome browser and its Network Inspector feature.
The only officially supported connectors and transports for shared resources are: HTTP/HTTPS, VM, JMS, JMS Caching Connection Factory, Database, WMQ, JBoss Transaction Manager, and Bitronix Transaction Manager.
When using MQ as a message processor with prefetch, only use a global prefetch configuration.
For example, the following local prefetch does not work:
1 2 3 4 <anypoint-mq:subscriber config-ref="Anypoint_MQ_Configuration" destination="programmatically" doc:name="Anypoint MQ" > <anypoint-mq:prefetch fetchSize="50" fetchTimeout="10000"/> </anypoint-mq:subscriber>
Use a global prefetch instead:
1 2 3 4 5 <anypoint-mq:prefetch name="Prefetch_Settings" fetchSize="50" fetchTimeout="10000" doc:name="Prefetch Settings"/> <anypoint-mq:subscriber config-ref="Anypoint_MQ_Configuration" destination="programmatically" doc:name="Anypoint MQ" prefetch-ref="Prefetch_Settings"/>
To delete a queue:
Click the right side of the queue entry in the Destinations table:
Click the trash can symbol in the upper right.
In the Delete Queue menu, click the checkbox:
Click Delete Queue.
Note: The time it takes to delete or purge a queue is approximately one minute. During this time, the status of the affected queue may not be updated.
To delete a message exchange:
Click the right side of the message exchange entry in the Destinations table:
Click the trash can symbol in the upper right.
In the Delete Exchange menu, click the checkbox:
Click Delete Exchange.