Limitations for Runtime Fabric on VMs / Bare Metal

The following information is important when installing, configuring, and managing Anypoint Runtime Fabric on VMs / Bare Metal.

Node, Environment, and Group Limitations

The following table lists important limitations for configuring Runtime Fabric:

Component	Limitation
Controller nodes	The maximum number of controller nodes is 5.
Worker nodes	The maximum number of worker nodes is 16.
Replicas per application	The maximum number of replicas per application is 8.
Replicas per worker node	The maximum number of replicas that can be deployed per worker node is 40. To allow core sharing across replicas when needed for bursting, run no more than 20-25 replicas per core. This ensures the Runtime Fabric’s internal components that run on each worker node are not overloaded by too many replicas.
Associated environments per Runtime Fabric	The maximum number of environments per Runtime Fabric is 50.
Business groups	You can create up to 50 Runtime Fabrics per org in a Business Group. Any sub org can contain up to 50 Runtime Fabrics, in addition to any shared by another sub org. For example, if you have parent Org A and its child Org B, you can have 50 Runtime Fabrics in Org A and 50 in Org B. You can also share all 50 Runtime Fabrics from Org A with Org B, and as a result, you will see 100 Runtime Fabrics in total in the list view of Org B.
Object store persistence	To handle data persistence across Mule application replicas and restarts, Anypoint Runtime Fabric uses the Persistence Gateway. See Persistence Gateway.

Component

Limitation

Controller nodes

The maximum number of controller nodes is 5.

Worker nodes

The maximum number of worker nodes is 16.

Replicas per application

The maximum number of replicas per application is 8.

Replicas per worker node

The maximum number of replicas that can be deployed per worker node is 40.

To allow core sharing across replicas when needed for bursting, run no more than 20-25 replicas per core. This ensures the Runtime Fabric’s internal components that run on each worker node are not overloaded by too many replicas.

Associated environments per Runtime Fabric

The maximum number of environments per Runtime Fabric is 50.

Business groups

You can create up to 50 Runtime Fabrics per org in a Business Group. Any sub org can contain up to 50 Runtime Fabrics, in addition to any shared by another sub org. For example, if you have parent Org A and its child Org B, you can have 50 Runtime Fabrics in Org A and 50 in Org B. You can also share all 50 Runtime Fabrics from Org A with Org B, and as a result, you will see 100 Runtime Fabrics in total in the list view of Org B.

Object store persistence

To handle data persistence across Mule application replicas and restarts, Anypoint Runtime Fabric uses the Persistence Gateway. See Persistence Gateway.

Internal Load Balancer Resource Allocation and Performance

The following table lists the approximate number of requests (averaging 10 KB) that can be served with a single replica of the internal load balancer based on the number of CPU cores. This information is based on the performance test results as described in Resource Allocation and Performance on Anypoint Runtime Fabric on VMs / Bare Metal.

vCPU Cores	Max Requests per Second (Connection Reuse)	Max Requests per Second (No Connection Reuse)
`1.00`	2000	175
`0.75`	1500	100
`0.50`	1000	50
`0.25`	100	10

The internal load balancer runs on the controller nodes of Runtime Fabric on VMs / Bare Metal. Configure the VM size based on the amount and type of inbound traffic. You can allocate only half of the available CPU cores on each VM to the internal load balancer. Refer to Resource Allocation and Performance on Anypoint Runtime Fabric for additional information.

Operating System Limitations

The following table lists known limitations for host operating systems in Runtime Fabric:

Operating System Limitation

Operating System	Limitation
RHEL 8	Do not set the optional environment variable `disable_selinux` to `false` in the `fabric.tf` script, or installation will fail. Do not set SELinux to enforcing mode.
RHEL 7	If you have enabled SELinux, third-party software running on the host may change policy settings from SELinux. You can set SELinux to enforcing mode if you don’t change default policies.
RHEL	Runtime Fabric only supports instances of RHEL with Logical Volume Manager (LVM) on Azure Bare Metal. Non-LVM instances of RHEL on Azure Bare Metal are not supported.

RHEL 8

Do not set the optional environment variable disable_selinux to false in the fabric.tf script, or installation will fail.
Do not set SELinux to enforcing mode.

RHEL 7

If you have enabled SELinux, third-party software running on the host may change policy settings from SELinux.
You can set SELinux to enforcing mode if you don’t change default policies.

RHEL

Runtime Fabric only supports instances of RHEL with Logical Volume Manager (LVM) on Azure Bare Metal. Non-LVM instances of RHEL on Azure Bare Metal are not supported.

How Antivirus and DPI Software Impacts Runtime Fabric Functionality

Anypoint Runtime Fabric has specific hardware and OS requirements. MuleSoft Support provides Support SLA and Severity Levels based on our validation and certification of the requirements specified in the installation prerequisites for Runtime Fabric on VMs / Bare Metal and Runtime Fabric on Self-Managed Kubernetes.

Some third-party software requires root access to the host. This software includes, but is not limited to, antivirus and DPI software, which has been found to interfere with the behavior and requirements of Runtime Fabric. In many support cases where MuleSoft Support has detected antivirus software, they have seen issues such as port blocking, node-to-node communication blocking, Docker default bridge deletion, and filesystem access issues that interfere with Runtime Fabric installation, upgrade, and normal operations.

In the event that abnormal behavior is observed with Runtime Fabric, you may be asked to disable any running third-party software (including anti-virus or DPI software) to determine if it is the cause of the behavior. If disabling such software restores proper Runtime Fabric functionality, this would demonstrate an implicit incompatibility between that software package and Runtime Fabric, and we would consider the support case resolved.

If you are strictly required to run traditional security tools that interfere with certain components of Runtime Fabric, vendors and security experts that support Kubernetes can support you in meeting this requirement. Cloud platform vendors often add cloud-centric controls that are consistent with well-known security benchmarks, such as the Center for Internet Security (CIS) Kubernetes Benchmark.