Contact Free trial Login

Configure Alerts for Anypoint Platform PCE

Anypoint Platform Private Cloud Edition (Anypoint Platform PCE) provides built-in alerts that are triggered when a condition specified in any alert definition is detected.

Measurements are stored in Influxdb and read by Kapacitor. Kapacitor sends emails when an alert is triggered.

Alert Definitions

Table 1. Alert Definitions
Name Description Threshold Config map

High CPU

Measures the total CPU usage on all cluster nodes

Warning 75%, Critical 90%

kapacitor-alerts

High Memory

Measures the memory consumption on all cluster nodes

Warning 75%, Critical 90%

kapacitor-alerts

Filesystem Usage

Measures the file system usage

Warning 80%, Critical 90%

kapacitor-alerts

Kubernetes Node Status

Checks Kubernetes status for all nodes

When Kubernetes reports a NotReady node status

kapacitor-alerts

Etcd Latency

Measures the latency between nodes, which can affect etcd performance

Warning 521 seconds, Critical 1024 seconds

kapacitor-alerts

Systemd health

Checks if the systemd slice (where gravity is running) is healthy

When gravity or child systems, such as docker, etcd, kube-proxy, kube-kubelet, planet-agent, dnsmasq, and flanneld, are down

kapacitor-alerts

Influxdb Healthcheck Status

Checks the health of Influxdb (where the metrics for monitoring are stored)

No connection to Influxdb

kapacitor-alerts

Networking Configuration

Checks networking operating system flag status

When ip_forward or br_netfilter flags are disabled

kapacitor-alerts

Node Uptime

Checks if nodes are online

When a node is down for more than five minutes

kapacitor-alerts

Etcd Health

Check the status of etcd

When etcd is down or not healthy

kapacitor-alerts

Docker Status

Verifies that the docker service is running on each node

When docker service is not running on a node

kapacitor-alerts

Backup Failing

Checks the result of a backup operation

When a backup operation fails

backup-status-alert

Stolon Replication Lag

Verifies that Postgres replicas are in sync

When a replica is lagging by 1800 or more seconds

kapacitor-alerts

Cassandra Load

Verifies that Cassandra nodes are in sync

Warning 10% difference, Critical 20% difference

cassandra-load

Configure Alert Definitions

  1. From OpsCenter, navigate to Configuration.

  2. Select the monitoring namespace.

  3. Select a config map and apply your changes.

If you apply changes to the kapacitor-alerts config map, you must restart Kapacitor. Follow the instructions in the Restart Kapacitor section.

Configure Alerts

  1. Configure the FROM and TO email addresses:

    1. Log in to the Ops Center console and then select Configuration.

    2. From the Namespace drop-down list, select monitoring.

    3. From the Config maps drop-down list, select alerting-addresses.

    4. Select From and then enter the FROM email address used for the alert.

    5. Select To and then enter the TO email address used for the alert.

  2. Configure your SMTP email server to send alerts.

    Configuring the SMTP server in Anypoint Access Management restarts the Anypoint Platform service responsible for initiating alerts.

  3. Verify alerts configuration:

    • Verify that your SMTP server can send and receive emails using the addresses you configure as the FROM and TO addresses in the following section.

    • Verify that your cluster nodes can communicate with your SMTP server.

      For example, use telnet to connect to your SMTP server from one of your cluster nodes:

      telnet my.smtp.server.com 587
      Trying XXX.XXX.XXX.XXX...
      Connected to my.smtp.server.com.
      Escape character is '^]'.
      220 my.smtp.server.com ESMTP
      ^[^]
      telnet> quit
      Connection closed.

Troubleshooting

If the connection is working, you can check Kapacitor logs for other errors when sending email alerts. On Ops Center, you can retrieve log information in the console ouput by running the following command:

kubectl logs -n monitoring -l component=kapacitor -c kapacitor

Restart Kapacitor

When you change the SMTP settings from Access Management, Kapacitor is automatically restarted to include the updated settings.

If you change an alert definition or the alerting-addresses config map without changing the SMTP settings from Access Management, Kapacitor must be manually restarted:

  1. Open a terminal to one of the master servers.

  2. Run the following command:

    `kubectl delete pod -n monitoring -l component=kapacitor`
  3. Wait for a kapacitor pod to run. Run the following command for verification:

    kubectl get pod -n monitoring -l component=kapacitor`

See Also

Was this article helpful?

💙 Thanks for your feedback!

Edit on GitHub