Update Nodes

Where possible, we changed noninclusive terms to align with our company value of Equality. We maintained certain terms to avoid any effect on customer implementations.

Before You Begin

Set up and test the backup procedure on all clusters.
Perform a backup before attempting node maintenance to preserve data in the event of a failure.

Cluster and Application Updates

You must perform the following steps on one Anypoint Platform Private Cloud Edition node at a time to minimize or eliminate component breakage during the update process. You must complete the entire process on each node before the next node is addressed. After each node is updated, verify the health status of all cluster components and applications before proceeding with the next node.

Verify Cluster Health Status

Verify that the cluster is healthy before starting the update process. Otherwise, many of the assumptions made in this process are not valid.

Verify the output from the following commands:
```
gravity status
etcdctl cluster-health
```
If the output from the previous commands mentions a degraded state or errors, fix all issues before proceeding.
Verify that all pods are running:
```
kubectl get pod -o wide --all-namespaces | grep -vi Running
```
If the output from the previous command lists any pod not in "Running" state, then fix all issues before proceeding.

Install Updates

Select a node on which to perform the following steps:

Retrieve the name of the target node:
```
kubectl get nodes
```

Drain the node:

kubectl drain --delete-local-data --ignore-daemonsets <node_name>

After the node is drained, identify the gravity unit name:
```
sudo systemctl list-units --plain | awk '/planet-master/{print $1}'
```
The previous command outputs a systemd unit name, which is used as the $unit_name value in the following step.

Stop the gravity unit:

systemctl stop $unit_name
systemctl status $unit_name

After the status subcommand indicates that the gravity unit is stopped, install the updates and reboot the server.

Perform Post-update Verification

Verify that all systems correctly restarted:
```
systemctl status --failed
```
The previous command should return zero units, which means that your system restart was successful and all units started. If any units failed to start, fix all issues before proceeding.

Check the gravity systemd units:

systemctl list-units '*gravity__gravitational*'

Check for errors in the logs by running journalctl.
If everything started correctly, run gravity enter and verify the results.

Check the cluster status again:

gravity status
etcdctl cluster-health
gravity get pod -o wide--all-namespaces | grep -vi Running

Rejoin the Node

Check the gravity Planet service and teleport status:
```
systemctl list-units '*gravity__gravitational*'
```
If the status of the gravity service is not loaded, active, or running, then you must enable it:
```
systemctl enable --now <name_of_the_gravity_unit>
```
Uncordon the node to set the target node to again schedule pods. This can take a few minutes to complete.
```
kubectl uncordon <node_name>
```
Verify cluster status:
```
gravity status
```
This command should display a healthy status for all nodes of the cluster. After you verify that the node has rejoined the cluster and is in healthy status, repeat the process for the next node in the cluster until all nodes are updated.