Rate Limiting Policy

When you apply the Rate Limiting policy to your API from the UI, you can configure the following parameters:

When you use the UI to add identifiers to the policy configuration, you can define groups of requests. The configured limits apply independently to each group. You can also use the identifier #[attributes.method] for one bucket per HTTP method in your Rate Limiting policy configuration.

An Identifier in the UI is a non-obligatory parameter. By default, the Identifier has no value. Based on whether you accept the default or provide a value, the policy performs in the following ways:

The following example shows the order of events that occur over a period of time using the identifier #[attributes.method] for a limit of 3 requests every 10 seconds:

In the example:

The rate-limiting engine, which is HTTP agnostic, depends solely on the resolution of the DataWeave expression. You can alter the Identifier expression in the UI to cover complex rate-limiting scenarios.

For example, you can configure a Rate Limiting policy with an identifier that uses one bucket for all Class A and Class C LAN requests and another bucket for everything else. The following image illustrates the second bucket in the previous sentence, which corresponds to 3 requests per 10 seconds quota with the DataWeave expression #[attributes.queryParam[‘customIdentifier’]] as the policy identifier:

In the example:

This configuration creates a false or a true bucket that corresponds to the locality of the IP that made the request. The false and true values correspond to the domain of boolean values and not HTTP.

Nevertheless, the policy works correctly because the engine treats the resolved expression as a String. In this case, the value is automatically cast from Boolean to String. You can explicitly define casting in DataWeave by adding output text/plain --- to your script.

Every identifier result has one algorithm. You must carefully create a DataWeave expression that does not return an unbound or a very large co-domain, which requires hosting the same number of algorithms in memory (at least a request for every possible identifier has to be made, because algorithms are created using the lazy creation strategy).

For example, suppose the DataWeave expression uses the IP address as the identifier in a Mule runtime engine instance that is public to the internet. If every public IPv4 IP on the internet makes a request to this Mule instance, there will be 3,706,452,992 algorithms running in a single Mule instance.

At an average of 250 bytes per algorithm, this amounts to approximately 1 terabyte in rate-limit algorithms. Therefore, use a DataWeave expression that resolves to a finite number of identifier to keep the resulting set as small as possible.

Consider the same configuration example of 3 requests and a window that starts exactly at 12:00:00, is reset every 10 seconds, and has a 2-node Mule cluster. Both nodes start and end their windows at the same time, and the cluster allows 3 requests per window in total:

Because the policy is clusterized, the whole cluster accepts three requests. If the clusterizable policy is turned off, the Mule cluster can accept six requests per window: that is, three requests per node.

To avoid distributed counters negatively impacting performance due to node synchronization needs, the policy uses caching mechanisms to predict the behavior and maximize throughput. However, some scenarios nevertheless result in higher latency, so be careful in your use of clusterizable configuration.

You might have decentralized processing in your environment, with the following setup:

In such a scenario, you do not need to run the policy in a clustered setup. Simply set the policy limits of the policy lower than the backend capacity of the weakest of the nodes. Additionally, a load balancer might be useful in case a node goes down.

Alternatively, you might have centralized processing in your environment, with the following setup:

In such a scenario, you do not need a cluster. However, you must then configure the policy to have a value lower than the maximum capacity of the backend.

If your environment does not include load balancers, use a cluster instead of a standalone instance to be certain that your nodes can manage varying levels of traffic. The Rate Limiting policy is designed to work under both balanced and unbalanced workloads. Because the backend does not receive any extra requests, its maximum capacity is not exceeded.

Configure your environment to use processing windows longer than one minute, to prevent the latency potential caused by information sharing among nodes in a cluster.

This configuration recommendation applies to both Rate Limiting and Rate Limiting-SLA policies in a cluster scenario.

You can configure your Rate Limiting policy to use windows that persist as long as days, months, and years. For example, suppose you want to allow your user to consume 1 million requests per year, but you cannot ensure that the node will be up the entire period or will need maintenance, which may result in restarting the Mule runtime engine.

The algorithm has been running for several months, so the client will lose critical information. Persistence solves this problem by periodically saving the current policy state. In case of a redeployment or a restart, the algorithms are recreated from the last known persisted state or started from a clean state.

Although persistence is enabled by default, you can turn it off by setting the following property to false:

throttling.persistence_enabled

You can also tweak the persistence frequency rate, which has a default of 10 seconds:

throttling.persistent_data_update_freq