Manage Queues
You can view and control any Virtual Machine (VM) or Staged Event-Driven Architecture (SEDA) queues within the flows of your deployed applications from the Queues tab in Runtime Manager.
Queues are available only for applications deployed to CloudHub workers. They are not available for applications that you deploy with the Runtime Manager to local servers. See Deployment Options for more details.
CloudHub automatically detects the queues from within your flow and lists them on the Queues tab.
Without making any changes in the underlying application, you can:
-
See the number of queued and in-flight messages on each queue.
-
View a snapshot of the queue throughput for the past 24 hours.
-
Clear all pending messages on a queue.
You can also enable data-at-rest encryption for your persistent queues to ensure that any shared application data written to a persistent queue is encrypted.
Use Cases
The Queues tab is useful for the following use cases:
-
You want runtime visibility into the queues in your application.
-
You are testing your application to observe whether messages are traveling through your queues as expected.
-
Your application is stuck processing a message and you want to flush the messages from only that queue to resolve the issue.
Enable Queues
To enable queues:
-
Deploy an application to CloudHub that includes one or more queues.
See VM Connector.
-
In Runtime Manager, click the application and click Manage Application.
-
Click the checkbox for Persistent queues.
-
If you are enabling this setting when deploying an application, click Deploy Application.
If you are updating an existing application, click Apply Changes.
View Queues
After you enable persistent queues, click Queues in the left navigation menu. CloudHub lists the queues in your application in the order they appear in your application:
The name of each queue is determined by the path attribute of the associated VM endpoint, or, in the case of a SEDA queue, from the flow name.
-
The Queued column shows the number of messages currently waiting on each of the queues in your application.
-
The In-Flight column indicates the number of messages being processed in the flow after that queue.
The in-flight messages are still on the queue until processing is complete, then they are deleted from the queue.
-
The Processed Messages column shows a thumbnail graph depicting the number of messages processed through each queue over the past 24 hours.
The Queued and In-Flight columns update every five seconds, and the Processed Messages graph updates every five minutes.
Clear Queues
To clear one or more queues, click the checkboxes for the queues to clear, then click Clear.
Runtime Manager presents a warning message that confirms which queues to clear and the duration this process takes to occur. Click Clear Queues to proceed.
After you confirm, the queues that you selected display a "Clearing queue in progress" message until the clearing process is complete:
Clearing a queue flushes all messages waiting in the queue and all in-flight messages. After the queue is cleared, the queue immediately begins accepting messages again.
Encrypt Data at Rest in Persistent Queues
You can enable data-at-rest encryption for all your persistent queues. By enabling this feature, you ensure that any shared application data written to a persistent queue is encrypted, allowing you to meet your security and compliance needs.
| If you enable encryption and then disable it, any messages that were previously encrypted become unreadable. |
Batch Component and Persistent Queues
If your Mule application is using a Batch component and persistent queues, you might see batch records being processed multiple times. All batch records are stored in Amazon SQS and the visibility of the message is set to 70 seconds by default. If your batch process takes longer than 70 seconds, batch processes might see the same message again and process it multiple times.
To avoid this issue, set the persistent.queue.min.timeout system parameter to a larger value in the Properties tab of the application Settings page.
For example, if your batch process takes 30 minutes to complete,
set the value to persistent.queue.min.timeout=2700000 milliseconds (45 Minutes).
The maximum value is 43000000 milliseconds (12 hours).
Limitations
When enabling persistent queues in your application, be aware of the following:
-
Persistent queues do not guarantee one-time-only message delivery. Duplicate messages might be sent. If one-time-only message delivery is critical for your use case, do not enable persistent queues.
-
If you clear a queue using the Queues tab, any new messages sent to the queue during the queue clearing process might be lost.
-
Enabling persistent queues has a performance implication. Putting a small message (50kb or less) on a queue can take 10-20 milliseconds (ms). Taking the same message off a queue can take 70-100ms.