Contact Us 1-800-596-4880
  1. Homepage
  2. Runtime Fabric (1.13)
  3. On Self-Managed Kubernetes

  4. Configuring Ingress

Configuring Ingress for Runtime Fabric on Self-Managed Kubernetes

This content refers to configuring ingress on Runtime Fabric for versions 1.10 and later. Refer to the documentation for versions 1.8 or 1.9 for previous ingress configuration instructions.

Anypoint Runtime Fabric on Self-Managed Kubernetes enables you to specify custom Ingress configurations using an ingress resource template. In a template, similar to a Kubernetes ingress spec, you can specify annotations, ingress class name, and HTTP and HTTPS rules.

If you do not apply an ingress resource template for an application, you can’t add ingress endpoints when you deploy it.

The ingress controller included with Google Kubernetes Engine (GKE) provisions a separate HTTP load balancer per application by default. Refer to Default Ingress Controller Behavior with Runtime Fabric on GKE for more information.

AWS Application Load Balancers have similar behavior unless you use the group.name annotation. Refer to Application load balancing on Amazon EKS for details.

Benefits of Custom Ingress Resource Templates

Using custom ingress resource templates in Runtime Fabric enables you to take advantage of native Kubernetes ingress configuration functionality, including support for:

  • Multiple ingress configurations per application

  • Multiple ingress controllers within the same Runtime Fabric instance

  • TLS and HTTPS configuration

  • Custom URL naming

  • URL parameter placeholders

How Ingress Resources Templates Work in Runtime Fabric

The following diagram provides an overview of how to use ingress resource templates in Runtime Fabric:

Diagram shows the ingress template workflow in Runtime Fabric

  1. An IT administrator configures an ingress controller.

  2. The IT admin creates or modifies an ingress resource template using the necessary parameters. Various ingress controllers operate differently. Review your ingress controller documentation and adjust your annotations accordingly.

  3. The IT admin applies the template using kubectl apply commands.

  4. From the applied template, Runtime Fabric creates placeholder URI domains and propagates the ingress configuration in Anypoint Runtime Manager as a template for all application deployments.

  5. A Mule application developer uses Runtime Manager to configure an application for deployment by selecting an available host, an optional subdomain, and a path combination, as specified in the ingress template.

  6. Runtime Fabric receives the application deployment request and uses the ingress configurations from the template to create a corresponding ingress object in the cluster.

Example Ingress Resource Templates

The following examples illustrate how to modify a Kubernetes ingress spec to create a Runtime Fabric ingress-resource template.

Use the API version appropriate to your Kubernetes version:

  • Kubernetes v1.19 or later:

    • networking.k8s.io/v1

    • networking.k8s.io/v1beta1

      For Kubernetes v1.22 and later, the extensions/v1beta1 and networking.k8s.io/v1beta1 API versions of Ingress are not available.

  • Kubernetes v1.18 and earlier:

    • only networking.k8s.io/v1beta1

networking.k8s.io/v1

This template is a simplified version of the configuration. The final configuration of the template can vary depending on the controller and ingress routing rules you use. Review the documentation for your ingress controller carefully.

 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example-rtf-ingress
  namespace: rtf (1)
  labels:
spec:
  ingressClassName: rtf-nginx (2)
  tls: (3)
  - hosts:
      - app-name.example.com
    secretName: example-tls
  rules:
  - host: app-name.example.com (4)
    http:
      paths: (5)
      - pathType: Prefix
        path: / (6)
        backend: (7)
          service:
            name: service-name
            port:
              name: service-port

Note the following about this example:

1 The template must be placed in the rtf namespace.
2 ingressClassName must be prefixed with rtf-, for example, rtf-nginx.

Runtime Fabric uses the rtf- prefix to recognize the object as a template. Templates with rtf- prefix in the spec.ingressClassName field or in the kubernetes.io/ingress.class annotation are consumed by the Runtime Fabric agent only and not by the actual ingress controller. Ingress controllers discover only those resources with a spec.ingressClassName value that use the vendor-specific name, for example, nginx or haproxy.
3 TLS is optional.

If you add the TLS section with a secretName attribute, you must ensure that you also create the referred TLS secret in the rtf namespace.
4 The app-name placeholder parameter is replaced with the actual app name when you deploy the application. It helps to ensure that each endpoint name is unique if there is not a wildcard in the subdomain.
5 A template can include multiple paths for a host, but Runtime Manager displays only the first path rule for the host.
6 The path parameter value is replaced by the value you add in the Path field when you configure ingress for a Mule application. Note that path is not a placeholder.
7 These placeholder values are required for Kubernetes validation, but the actual values are not used by Runtime Fabric.

For a path-based routing ingress template, consult your ingress controller documentation and configure it accordingly. For an example using Nginx, see Path-based Nginx Ingress Routing Example for Runtime Fabric on Self-Managed Kubernetes.

networking.k8s.io/v1beta1

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: example-rtf-ingress
  namespace: rtf (1)
  labels:
spec:
  ingressClassName: rtf-nginx (2)
  tls: (3)
  - hosts:
      - app-name.example.com
    secretName: example-tls
  rules:
    - host: app-name.example.com (4)
      http:
        paths: (5)
        - path: / (6)
          pathType:
          backend: (7)
            serviceName: service-name
            servicePort: 80

Note the following about this example:

1 The template must be placed in the rtf namespace.
2 ingressClassName must be prefixed with rtf-, for example, rtf-nginx.

Runtime Fabric uses the rtf- prefix to recognize the object as a template. Templates with rtf- prefix in the spec.ingressClassName field or in the kubernetes.io/ingress.class annotation are consumed by the Runtime Fabric agent only and not by the actual ingress controller. Ingress controllers discover only those resources with a spec.ingressClassName value that use the vendor-specific name, for example, nginx or haproxy.
3 TLS is optional.

If you add the TLS section with a secretName attribute, you must ensure that you also create the referred TLS secret in the rtf namespace.
4 The app-name placeholder parameter is replaced with the actual app name when you deploy the application. It helps to ensure that each endpoint name is unique if there is not a wildcard in the subdomain.
5 A template can include multiple paths for a host, but Runtime Manager displays only the first path rule for the host.
6 The path parameter value is replaced by the value you add in the Path field when you configure ingress for a Mule application. Note that path is not a placeholder.
7 These placeholder values are required for Kubernetes validation, but the actual values are not used by Runtime Fabric.

For a path-based routing ingress template, consult your ingress controller documentation and configure it accordingly. For an example using Nginx, see Path-based Nginx Ingress Routing Example for Runtime Fabric on Self-Managed Kubernetes.

Template Placeholders

The following placeholders are optional. If you use a placeholder, replace it in the appropriate location in the template. Placeholders must be lowercase.

Name of Placeholder Location Example Value

business-group-id

path, host, annotation, labels

 
labels:
  business:
  business-group-id

environment-id

path, host, annotation, labels

 
labels:
  environment:
  environment-id

unique-id

path, annotation, labels

 
path: /app-name-unique-id.com

Use the unique-id placeholder to create a unique application URL. This is useful if you want to deploy two applications with same name from different business groups. For example, if you use app-name-unique-id.com in the path section of the ingress template, the resulting endpoint will be similar to app-name-7jkbic.com.

The unique-id placeholder is available in Runtime Fabric version 1.12.28 and later.

Using TLS in the Ingress Template

To use TLS in the ingress template, you must create a TLS secret. See the Kubernetes TLS documentation for details.

You can use the rtf.mulesoft.com/synchronized=true label to configure Runtime Fabric to synchronize any TLS secret updates to all application namespaces, for example:

apiVersion: v1
kind: Secret
metadata:
  name: testsecret-tls
  namespace: default
  labels:
    rtf.mulesoft.com/synchronized=true
data:
  tls.crt: base64 encoded cert
  tls.key: base64 encoded key
type: kubernetes.io/tls

Applying this label to your secret enables your ingress controller to immediately apply the updated TLS material into application namespaces. If you don’t add this label, you must manually update the secret in each namespace.

Configure Ingress for a Mule Application Deployment in Runtime Fabric

To configure ingress, complete the following tasks:

  1. Create an ingress resource template and apply it in the cluster.

  2. Configure ingress for a Mule application.

Before You Begin

Before configuring ingress in Runtime Fabric, you must set up an ingress controller. See the Kubernetes documentation for a list of ingress controllers.

Create an Ingress Resource Template and Apply it in the Cluster

Use one of the example templates to create an ingress resource.

  1. Copy the example template into a new file and modify it according to the notes.

  2. Include a .yaml extension in the file name.

  3. To apply an ingress template, run:

    kubectl apply -f <TEMPLATE_FILENAME.yaml>

If there is an issue in the template, Kubernetes api-server returns an error and the command fails.

After you apply a template successfully, you can view it with:

kubectl describe ingress [Ingress Name] -n rtf

Configure Ingress for a Mule Application

You configure ingress for an application when you deploy it to Runtime Fabric using Runtime Manager. Available hosts and paths for an application come from the ingress resource template configured by the Runtime Fabric administrator.

You can also use this procedure to deploy a test application to validate your ingress resource template.

The following provides an overview of how to configure ingress for an application. Refer to Deploy a Mule Application to Runtime Fabric for complete deployment instructions. If you are using the Mule Maven plugin to deploy applications to Runtime Fabric, the deployment settings parameter publicUrl in the block http can accept a comma-delimited string of multiple endpoints. For more information, see deploymentSettings Parameters Reference.

  1. Navigate to Runtime Manager and follow the documentation to deploy an application to Runtime Fabric.

  2. Select Ingress.

  3. From the Host drop-down list, select a host for the application.

  4. If the hostname uses a wildcard, add a subdomain in the Subdomain field.

    The Subdomain field is only available if the hostname uses a wildcard.

  5. In the Path field, add a URL path to the application’s endpoint:

    The host and path fields are populated for the endpoint

  6. To preview the endpoint, click the generated preview link.

  7. To add additional endpoints, click + Add Endpoint.

  8. When ready, click Deploy application.

Existing Ingress Configurations Translated Post-Upgrade

When upgrading to Runtime Fabric version 1.10, the upgrade process automatically translates any existing ingress ConfigMaps to the ingress resource template format, including annotations, path, namespace, and host parameters. Review the table below to determine if any action is required after upgrading.

Runtime Fabric Pre-upgrade Runtime Fabric Upgraded Action Required TLS Support

No domains configured and no ingress-ConfigMap applied

No template is generated automatically after the upgrade

Create a template. See Create an Ingress Resource Template and Apply it in the Cluster.

-

No domains configured, but a valid ingress-ConfigMap applied

Generated template will contain * for hosts and will contain the path and annotations from the previous ingress-ConfigMap

Edit the host field in the generated template to provide a valid host/domain for deploying applications in Runtime Manager

No TLS section added by default

Valid domains configured, but no ingress-ConfigMap applied

Generated templates will contain one host per domain per template and will include /app-name in the path for each template

No immediate action is required unless the domains use https://; in this case, configure a TLS secret in the template corresponding to the HTTPS domain

For domains that have https://, the template corresponding to that domain will have a TLS section without a secret attribute

Valid domains configured and a valid ingress-ConfigMap applied

Generated templates will contain one host per domain per template and will include all the paths and annotations from the previous ingress-ConfigMap

No immediate action required, unless the domains use https://; in this case, configure a TLS secret in the template corresponding to the HTTPS domain

For domains that have https://, the template corresponding to that domain will have a TLS section without a secret attribute

Troubleshooting Ingress Issues

If you encounter errors when attempting to use ingress for Runtime Fabric, troubleshoot them as follows.

Mule App Deployed But Cannot Access Its Endpoint

Scenario: You successfully deploy a Mule application in Runtime Manager, but you cannot access the application’s endpoint.

To troubleshoot this issue:

  1. Verify that the application is listening on port 8081:

    kubectl port-forward -n [NAMESPACE] svc/<APP_NAME> 8081:8081

  2. Verify that the application is running and responding to HTTP requests:

    curl -v http://127.0.0.1:8081/

    This helps determine if the API access issue is on the Mule app itself.

  3. Validate that an ingress resource exists for that application service:

    kubectl get ingress -n [NAMESPACE]

  4. Validate that the service was created:

    kubectl get svc -n [NAMESPACE]

    If not, check the Runtime Fabric agent logs:

    kubectl logs -n rtf [AGENT_POD_NAME] -f

If the service and ingress object look healthy, see additional troubleshooting tasks.

Ingress Resource Created in Cluster But Cannot Access Its Endpoint

Scenario: Runtime Fabric successfully creates an ingress resource in the cluster but you cannot access the application’s endpoint due to a 404 error.

To troubleshoot this issue:

  1. Verify the ingress and service resources:

    kubectl get ingress -n<APP-NAMESPACE>
kubectl get svc -n<APP-NAMESPACE>

    The results should be similar to:

    # kubectl get ingress -n<<APP-NAMESPACE>
NAME                           CLASS       HOSTS               ADDRESS       PORTS     AGE
<INGRESS_RESOURCE>            <INGRESS>    <HOSTNAME>.com      <HOST IP>     80, 443   7m3s

#kubectl get svc -n<<APP-NAMESPACE>
NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
<SERVICE-NAME>       ClusterIP   <CLUSTER IP>   <none>        8081/TCP,8082/TCP,5701/TCP   8m5s

  2. Check the ingress resource for your app to ensure that the HTTP path in the resource, annotations, and host are correctly rendered:

    kubectl get ing -n<APP-NAMESPACE> <INGRESS-RESOURCE-NAME> -oyaml

    The results should be similar to:

    # kubectl get ing -n<APP-NAMESPACE> <INGRESS-RESOURCE-NAME> -oyaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  labels:
    ingress.rtf.mulesoft.com/svcName: <APP_NAME>
    ingress.rtf.mulesoft.com/svcVersion: <UUID>
…
…
spec:
  ingressClassName: <INGRESS-CLASS>
  rules:
  - host: myapp.example.com
    http:
      paths:
      - backend:
          serviceName: myapp
          servicePort: 8081
        path: /
        pathType: Prefix
  tls:
  - hosts: myapp.example.com
    secretName: example-tls
status:
  loadBalancer:
    ingress:
    - hostname: <hostname>

  3. Check the application pod logs to ensure that you’ve configured the correct listening port:

    kubectl logs -f -n<APP-NAMESPACE> <APP-POD-NAME> -c app

    The results should be similar to:

    # kubectl logs -f -n<<APP-NAMESPACE> <APP-POD> -c app
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ Starting app '<APP_NAME>'                                                     +
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[2021-07-29 21:22:37.842] INFO  QueueXaResourceManager [ArtifactDeployer.start.01] [event: ]: Starting ResourceManager
[2021-07-29 21:22:37.842] INFO  QueueXaResourceManager [ArtifactDeployer.start.01] [event: ]: Started ResourceManager
[2021-07-29 21:22:37.846] INFO  AbstractLifecycleManager [ArtifactDeployer.start.01] [event: ]: Starting Bean: org.mule.runtime.module.extension.internal.runtime.config.ConfigurationProviderToolingAdapter-HTTP_Listener_config
[2021-07-29 21:22:37.859] INFO  GrizzlyHttpServer [ArtifactDeployer.start.01] [event: ]: Listening for connections on 'http://0.0.0.0:8081'
[2021-07-29 21:22:37.874] INFO  FlowConstructLifecycleManager [ArtifactDeployer.start.01] [event: ]: Starting flow: sample-json-backendFlow
[2021-07-29 21:22:38.171] INFO  AbstractLifecycleManager [ArtifactDeployer.start.01] [event: ]: Starting Bean: listener
[2021-07-29 21:22:38.178] INFO  LogUtil [ArtifactDeployer.start.01]:

  4. In the logs, verify that the listener port matches the service port discovered in Step 1.

  5. If the port is correct, check the application logs to verify that the application is receiving requests from the ingress controller.

Ingress Resource Not Recognized by AWS Application Load Balancer

Scenario: When using an AWS ALB, you can’t access an application’s endpoint, even though you successfully deployed the application and endpoint.

If you’re using AWS Load Balancer Controller for ingress, you must specify the kubernetes.io/ingress.class: rtf-alb annotation in the template, not ingressClassName: rtf-alb. AWS Load Balancer Controller requires the ingress.class annotation to discover and create L7 load balancers for deployed ingress resources for those annotations.