Contact Free trial Login

Configure Queues

A queue is a temporary storage area for a message. You can create queues, FIFO queues, and dead-letter queues in Anypoint Platform. You can also send and receive messages from a queue in Anypoint Studio.

Using Anypoint MQ, you can create two types of queues:

  • Standard queue

    These queues don’t guarantee a specific message order. Standard queues are the best fit for applications in which messages must be delivered quickly.

    Anypoint MQ supports up to 120,000 in-flight messages per standard queue.

  • FIFO (first in, first out) queue

    These queues ensure that your messages arrive in order. FIFO queues are the best fit for applications requiring strict message ordering and exactly-once delivery, but in which message delivery speed is of less importance. See FIFO Queues.

    Anypoint MQ supports 10 in-flight messages per message group in a FIFO queue.

You can assign either standard or FIFO queue type as a dead-letter queue (DLQ). See Dead Letter Queues.

Standard queues can be bound to a message exchange, enabling you to send messages to multiple queues.

You can filter queues by type in the Destinations page:

Filters at the top of the Destinations page
Figure 1. The arrow shows the filters at the top of the Destinations page.

Prerequisites

You must create an environment, user access, and roles for accessing Anypoint MQ before configuring queues. See Configure Environment, User, and Role Access.

Create a Queue

To create a standard queue or FIFO queue:

  1. Log in to Anypoint Platform.

  2. Click MQ in the left navigation bar or the main Anypoint Platform screen.

  3. Click Destinations.

  4. Click the blue plus icon (+) to display the menu:

    Blue plus icon used for adding a queue
    Figure 2. The arrow shows the blue plus icon at the top of the Destinations page.
  5. Select Queue or FIFO Queue.

    For Exchange, see Configure Message Exchanges.

  6. Complete the fields in Create Queue or Create FIFO Queue:

    Create Queue popup
    Figure 3. The screenshot shows the Create Queue popup.
    • ID specifies the queue name.

      Queue names can contain up to 127 alphanumeric characters (a-z, A-Z, 0-9), periods (.), and hyphens (-). They cannot contain spaces or other characters.

    • Message TTL (Time-to-live) specifies how long unprocessed messages persist before being deleted.

      The maximum TTL for a message is two weeks.

    • Default Acknowledgment Timeout specifies how long a message waits before being put back into the queue in the event of server failure when the message is not acknowledged.

      The lock makes the message unavailable to other apps while locked, but does not block other messages from being read.

      The maximum acknowledgement timeout is 12 hours.

    • Assign Default Delivery Delay specifies how long to delay delivery for messages sent to the queue.

      You can specify to delay message delivery in seconds, minutes, hours, or days. When you send a message, you can override the default delivery delay. See Delay queues.

    • Encryption encrypts all messages in the queue.

    • Assign a Dead Letter Queue assigns the DLQ.

      The DLQ is a previously created queue to which to send undeliverable messages. See Dead Letter Queues.

  7. Click Create Queue.

    The queue is created and appears in the Destinations page:

    Destinations page with new queue
    Figure 4. The screenshot shows (1) the ID, (2) the type, and (3) the details label for the queue in the Destinations page.

    From this page, you can:

    • Click the queue type for the new queue to display its details in the right pane.

    • Click the queue ID or details label to display the Queue Settings page.

Send a Message to a Queue

Message content can be text, JSON, or CSV (comma-separated values). The maximum size of a message is 10 MB.

To send a text message to a queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue ID to display the Queue Settings page.

  3. In the settings page, click Message Sender in the left pane:

    Message Sender option
    Figure 5. The arrow shows the Message Sender option in the left pane.
    • Type is Text, CSV, or JSON.

    • Payload contains the message payload text, such as Hello Mules!.

    • Delivery Delay (standard queue only) indicates the time to delay the message delivery (optional).

      If you specified Assign Default Delivery Delay on the standard queue, you see the default delivery delay. You can override that value for this message or set a delivery delay for a message in a queue without delivery delay.

      Delayed delivery is not supported for individual messages in a FIFO queue.

    • Message Group ID (FIFO queue only) specifies the group to which the message belongs.

      Message group IDs can contain up to 127 alphanumeric characters (a-z, A-Z, 0-9), periods (.), and hyphens (-). They cannot contain spaces or other characters.

      To preserve message order for an app, you might want to create a message group for each application to send messages:

      Message Group ID field for FIFO queues in the Message Sender page
      Figure 6. The arrow shows the Message Group ID field in the Message Sender page.
    • Add User Properties configures user properties to send with the message.

      Toggle Add User Properties and specify the property name-value pairs to send:

      Add User Properties toggle in the Message Sender page
      Figure 7. The arrow shows the Add User Properties toggle in the Message Sender page.

      User property names can contain up to 42 alphanumeric characters (a-z, A-Z, 0-9), periods (.), and hyphens (-). They cannot contain spaces or other characters.

  4. Click Send.

View the Number of Messages in a Queue

You can verify that a message arrived in the queue by viewing the number of messages in the queue from the Destinations page.

To see the number the messages in the queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue type for the queue to display its details in the right pane.

    The details pane shows the number of messages in the queue:

    The number of messages in the queue in the details pane
    Figure 8. The arrow shows the number of messages in the queue in the details pane.

Get a Message from a Queue

You can verify that a message arrived by retrieving it from the queue.

To get a message from a queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue ID to display the Queue Settings page.

  3. In the settings page, click Message Browser in the left pane:

    Message Browser option and Get Messages button
    Figure 9. The screenshot shows (1) the Message Browser option and (2) the Get Messages button.
  4. Specify the number of messages (1-500) and the time to poll (1-3600 seconds).

    For a FIFO queue, you can retrieve up to 10 messages.

  5. If you are retrieving messages from a FIFO queue, select the checkbox to acknowledge that viewing messages through the browser might cause other consumers of the queue to be blocked from accessing messages in the queue:

    Accept checkbox for FIFO queues
    Figure 10. The arrow shows the accept checkbox for getting messages from a FIFO queue.
  6. Click Get Messages.

  7. Click the message row to view the message payload in the details pane:

    Selected message row
    Figure 11. The screenshot shows (1) the message row, (2) the message payload in the details pane, (3) the Return Messages icon, (4) the Delete icon, (5) the Full Screen icon, (6) the close details pane icon (X), and (7) the user properties.
    • Click Full Screen to display the message payload in a separate window.

    • Click X to close the details pane.

    • Click Delete to delete the selected message.

      Deleting a message is an acknowledgment operation (ACK). For information about how Anypoint MQ processes ACK messages, see Acknowledge Messages.

    • Click Return Messages to return all messages to the queue (for example, to enable other applications to read the messages).

      Returning messages to the queue is a not-acknowledgment operation (NACK). The messages aren’t altered. However, the time-to-live (TTL) value you set when you created the queue determines how long the message is available before Anypoint MQ deletes it.

      If you switch from the Message Browser page, messages automatically return to the queue.

      If user properties are configured in the Message Sender page or as part of the Publish operation in Anypoint MQ Connector, they appear in the details pane for the message:

User Properties in Message Browser details pane
Figure 12. The arrow shows the User Properties in the details pane.

Send a JSON Message to a Queue

To send a JSON message to a queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue ID to display the Queue Settings page:

    Queue Settings option
    Figure 13. The arrow shows the Queue Settings option in the Destinations page.
  3. In the settings page, click Message Sender in the left pane.

  4. Set the Type to JSON.

  5. In the Payload field, enter this text:

    {
      "animal that walks": "dog",
      "animal that swims": "fish",
      "animal that flies": "parrot"
    }

    For Delivery Delay (standard queue only) and Message Group ID (FIFO queue only), see Send a Message to a Queue.

  6. Click Send.

  7. Click Message Browser, and then click Get Messages.

  8. Click the message row to view the message payload in the details pane:

    JSON payload
    Figure 14. The screenshot shows the JSON message payload, (1) the Full Screen icon, and (2) the close details pane icon (X).

    Click Full Screen to display the message payload in a separate window or X to close the details pane.

Send a CSV Message to a Queue

To send a CSV message to a queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue ID to display the Queue Settings page:

    Queue Settings option
    Figure 15. The arrow shows the Queue Settings option in the Destinations page.
  3. In the settings page, click Message Sender in the left pane.

  4. Set the Type to CSV.

  5. In the Payload field, enter this text:

    "dog",
    "fish",
    "parrot"

    For Delivery Delay (standard queue only) and Message Group ID (FIFO queue only), see Send a Message to a Queue.

  6. Click Send.

  7. Click Message Browser, and then click Get Messages.

  8. Click the message row to view the message payload in the details pane:

    CSV payload
    Figure 16. The screenshot shows the CSV message payload, (1) the Full Screen icon, and (2) the close detail pane icon (X).

    Click Full Screen to display the message payload in a separate window or X to close the details pane.

Purge Messages from a Queue

To purge all the messages in the queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue type for the queue to display its details in the right pane.

  3. Click the Purge Messages icon in the details pane:

    Purge Messages icon in the details pane
    Figure 17. The arrow shows the Purge Message icon in the details pane.
  4. In the Purge Messages confirmation box, select the checkbox and click Delete Messages to verify that you want to purge all messages in the queue:

    Accept checkbox for purging messages from a queue
    Figure 18. The arrow shows the accept checkbox for purging messages from the queue.

Delete a Queue

If a queue is no longer needed, you can delete it. Any in-flight messages in the deleted queue are lost. You can’t recover a deleted queue.

To delete a queue:

  1. In Anypoint Platform > MQ, click Destinations in the left pane.

  2. Click the queue type for the queue to display its details in the right pane.

  3. Click the Delete icon in the details pane:

    Delete icon in the details pane for deleting a queue
    Figure 19. The arrow shows the Delete icon in the details pane for deleting a queue.
  4. In the Delete Queue confirmation box, select the checkbox and click Delete Queue to verify that you want to delete the queue:

    Accept checkbox for deleting a queue
    Figure 20. The arrow shows the accept checkbox for deleting a queue.

    The time it takes to delete or purge a queue is approximately one minute. During this time, the status of the affected queue can’t be updated.

FIFO Queues

For applications where the order of messages must be strictly preserved and enforced, Anypoint MQ provides first in, first out (FIFO) processing to enable message ordering. With FIFO queues, Anypoint MQ ensures that the order in which messages are placed in a queue is the same order messages are retrieved.

FIFO queues have the following benefits and restrictions:

  • Are limited to 300 TPS (transactions per second)

    However, if you batch 10 messages per each read and write operation (maximum) using the API, FIFO queues can support up to 3,000 TPS.

  • Support delivery delay on all messages in the FIFO queue

    If you change the delivery delay for a FIFO queue, the setting change is retroactive to all messages in the queue. Delayed delivery is not supported for individual messages in a FIFO queue.

  • Support multiple message groups

  • Provide encryption

    Encrypting a FIFO queue does not affect the order or the contents of messages consumed.

  • Can be used as a DLQ

    When assigning a DLQ to a FIFO queue, the DLQ must also be a FIFO queue.

  • Cannot be bound to a message exchange

  • Should not be used with prefetch mode.

To create a FIFO queue, see Create a Queue. To determine if a queue is FIFO or standard (non-FIFO), view its details.

See the Anypoint MQ FAQ for the regions in which Anypoint MQ and FIFO queues are available.

For information about creating FIFO queues with the Admin API, see Create a FIFO Queue in the Admin API Portal.

FIFO Queues and Message Groups

Message groups enable multiple consumers to access messages in a FIFO queue. Only one consumer at a time can access messages in a message group. When a consumer is accessing a message group, other message groups in the FIFO queue are available to other consumers, reducing the chance that the FIFO queue is blocked.

The FIFO queue processes messages in the message group one by one, in the order they were added. If the queue includes multiple message group IDs, it returns as many messages with the same message group ID as possible.

FIFO queues do not support retrieving messages by message group ID.

Message order is preserved within the message group, but is not enforced across message groups. If multiple applications send messages associated with the same message group ID to a FIFO queue, the messages are stored in the order they arrive. To preserve message order for an app, make sure that each app uses a specific message group ID to send messages.

If a message doesn’t have an associated message group ID, the message is assigned to a default message group. If you require a single group of ordered messages, provide the same message group ID for messages sent to the FIFO queue.

Assign a Message Group ID to a Message

To assign a message group ID to a message:

  1. Send a message to a queue.

  2. In the Message Sender page, specify the message group to which the message belongs:

    Message Group ID field for FIFO queues in the Message Sender page
    Figure 21. The arrow shows the Message Group ID field in the Message Sender page.

Determine the Message Group to Which a Message Belongs

To see which message group a message belongs to:

  1. Get a message from the queue.

  2. In the Message Browser, check the details pane for the message:

    Message Group ID value in the details pane for a message
    Figure 22. The arrow shows the Message Group ID value in the details pane for a message in the queue.

FIFO Exactly-Once Delivery

For applications in which messages must be processed exactly once, such as those used in transactional use cases, FIFO queues ensure exactly-once message delivery.

FIFO queues provide message deduplication. If multiple messages are sent to a FIFO queue containing the same message ID, the first message to arrive is kept as the valid message. Subsequent messages within the deduplication interval of five minutes are considered duplicates and are discarded. For example, after the queue sees that a message with ID 12345 arrives in the queue, for the next five minutes, the FIFO queue discards any messages with the ID 12345.

When building applications in Anypoint Studio that require exactly-once delivery, you can either:

  • Set the message ID in the publisher settings in the Anypoint MQ connector.

  • Let Anypoint MQ autogenerate a unique message ID for each message sent to a queue (default).

Dead Letter Queues

Anypoint MQ enables you to ensure that messages that aren’t delivered are sent to a queue known as the dead-letter queue (DLQ). You can then analyze the messages sent to the DLQ to determine why those messages were not delivered.

A DLQ is essentially the same as any other queue except that it receives only undelivered messages. You can specify the time-to-live (TTL) value, encryption, and delivery delay when you create the queue.

You must have at least two queues for one of them to be assigned as a DLQ. Both queues must be:

  • The same type (standard or FIFO)

  • In the same geographical region

  • In the same environment and owned by the same Anypoint Platform account.

Encryption

Undeliverable messages that are sent to the DLQ use the encryption set for the source queue, regardless of the DLQ encryption setting. Messages that are sent directly to the DLQ by a client use the encryption setting for the DLQ. If your organization’s policy is that all messages are encrypted, then you must set encryption for all queues.

Billing and Maximum Deliveries

If a message sent to the DLQ is a dead letter from another queue, the message is not charged against the billable API requests. However, if a client sends a message directly to a DLQ, that message is charged.

If a queue has a dead-letter queue assigned:

  • Viewing the messages in the source queue using the Anypoint MQ Message Browser counts against the number of maximum deliveries.

  • Viewing a message and returning it to the queue counts as a NACK operation and is considered an unsuccessful delivery attempt.

  • Deleting the message in the Message Browser prevents it from being counted against the maximum deliveries, but the message is lost.

How Messages Are Sent to a DLQ

This flowchart shows how messages are sent to a DLQ:

mq dlq flowchart

Assign a DLQ to a Queue

You can assign any queue as a DLQ for another queue as long as it’s the same type and in the same region and environment.

  • Messages sent to a FIFO DLQ must originate from a FIFO queue.

  • Messages sent to a standard DLQ must originate from a standard queue.

To assign a DLQ to a queue:

  1. Create a queue.

  2. Toggle Assign a Dead Letter Queue to On.

    The following additional fields appear:

    DLQ fields in the Create Queue page
    Figure 23. The screenshot shows the DLQ fields in the Create Queue page.
    • Dead Letter Queue Name

      Choose a previously created queue name from the drop-down list.

    • Reroute after 10 attempts (Optional)

      Specify how many times Anypoint MQ attempts to deliver messages in the queue before rerouting the message to the dead-letter queue. If not specified, the default value is 10 tries. This value ranges from 1 to 1000 attempts.

Recover Messages from a DLQ

You can recover messages from the DLQ using the REST API to get the message from the queue and writing the message to a new queue. See Anypoint MQ Admin API.

Determine Whether a Queue is a DLQ

You can see whether a queue is a DLQ by viewing its details:

DLQ status in the details pane
Figure 24. The screenshot shows the DLQ status in the details pane for the queue.

You can also view details of each queue from the REST Administration API, using the Get Queue REST endpoint.

{
 "encrypted": false,
 "type": "queue",
 "queueId": "my-dlq-1",
 "deadLetterSources": [
   "my-queue-4",
   "my-dls-1"
 ],
 "defaultTtl": 2000000,
 "defaultLockTtl": 2000000
}

If DLQ is set, the returned entities contain the deadLetterSources field.

For more information, see Anypoint MQ Admin API.

Was this article helpful?

💙 Thanks for your feedback!

Edit on GitHub