$ oc adm cordon <node1>
As an administrator, you can perform a number of tasks to make your clusters more efficient.
Evacuating pods allows you to migrate all or selected pods from a given node or nodes.
You can only evacuate pods backed by a replication controller. The replication controller creates new pods on other nodes and removes the existing pods from the specified node(s).
Bare pods, meaning those not backed by a replication controller, are unaffected by default. You can evacuate a subset of pods by specifying a pod-selector. Pod selectors are based on labels, so all the pods with the specified label will be evacuated.
Mark the nodes unschedulable before performing the pod evacuation.
Mark the node as unschedulable:
$ oc adm cordon <node1>
node/<node1> cordoned
Check that the node status is Ready,SchedulingDisabled
:
$ oc get node <node1>
NAME STATUS ROLES AGE VERSION
<node1> Ready,SchedulingDisabled worker 1d v1.24.0
Evacuate the pods using one of the following methods:
Evacuate all or selected pods on one or more nodes:
$ oc adm drain <node1> <node2> [--pod-selector=<pod_selector>]
Force the deletion of bare pods using the --force
option. When set to
true
, deletion continues even if there are pods not managed by a replication
controller, replica set, job, daemon set, or stateful set:
$ oc adm drain <node1> <node2> --force=true
Set a period of time in seconds for each pod to
terminate gracefully, use --grace-period
. If negative, the default value specified in the pod will
be used:
$ oc adm drain <node1> <node2> --grace-period=-1
Ignore pods managed by daemon sets using the --ignore-daemonsets
flag set to true
:
$ oc adm drain <node1> <node2> --ignore-daemonsets=true
Set the length of time to wait before giving up using the --timeout
flag. A
value of 0
sets an infinite length of time:
$ oc adm drain <node1> <node2> --timeout=5s
Delete pods even if there are pods using emptyDir
volumes by setting the --delete-emptydir-data
flag to true
. Local data is deleted when the node
is drained:
$ oc adm drain <node1> <node2> --delete-emptydir-data=true
List objects that will be migrated without actually performing the evacuation,
using the --dry-run
option set to true
:
$ oc adm drain <node1> <node2> --dry-run=true
Instead of specifying specific node names (for example, <node1> <node2>
), you
can use the --selector=<node_selector>
option to evacuate pods on selected
nodes.
Mark the node as schedulable when done.
$ oc adm uncordon <node1>
You can update any label on a node.
Node labels are not persisted after a node is deleted even if the node is backed up by a Machine.
Any change to a |
The following command adds or updates labels on a node:
$ oc label node <node> <key_1>=<value_1> ... <key_n>=<value_n>
For example:
$ oc label nodes webconsole-7f7f6 unhealthy=true
You can alternatively apply the following YAML to apply the label:
|
The following command updates all pods in the namespace:
$ oc label pods --all <key_1>=<value_1>
For example:
$ oc label pods --all status=unhealthy
By default, healthy nodes with a Ready
status are
marked as schedulable, which means that you can place new pods on the
node. Manually marking a node as unschedulable blocks any new pods from being
scheduled on the node. Existing pods on the node are not affected.
The following command marks a node or nodes as unschedulable:
$ oc adm cordon <node>
For example:
$ oc adm cordon node1.example.com
node/node1.example.com cordoned
NAME LABELS STATUS
node1.example.com kubernetes.io/hostname=node1.example.com Ready,SchedulingDisabled
The following command marks a currently unschedulable node or nodes as schedulable:
$ oc adm uncordon <node1>
Alternatively, instead of specifying specific node names (for example, <node>
), you can use the --selector=<node_selector>
option to mark selected
nodes as schedulable or unschedulable.
When you delete a node using the CLI, the node object is deleted in Kubernetes, but the pods that exist on the node are not deleted. Any bare pods not backed by a replication controller become inaccessible to OKD. Pods backed by replication controllers are rescheduled to other available nodes. You must delete local manifest pods.
To delete a node from the OKD cluster, edit the appropriate MachineSet
object:
If you are running cluster on bare metal, you cannot delete a node by editing
|
View the machine sets that are in the cluster:
$ oc get machinesets -n openshift-machine-api
The machine sets are listed in the form of <clusterid>-worker-<aws-region-az>.
Scale the machine set:
$ oc scale --replicas=2 machineset <machineset> -n openshift-machine-api
Or:
$ oc edit machineset <machineset> -n openshift-machine-api
You can alternatively apply the following YAML to scale the machine set:
|
For more information on scaling your cluster using a machine set, see Manually scaling a machine set.
For more information on scaling your cluster using a MachineSet, see Manually scaling a MachineSet.
When you delete a node using the CLI, the node object is deleted in Kubernetes, but the pods that exist on the node are not deleted. Any bare pods not backed by a replication controller become inaccessible to OKD. Pods backed by replication controllers are rescheduled to other available nodes. You must delete local manifest pods.
Delete a node from an OKD cluster running on bare metal by completing the following steps:
Mark the node as unschedulable:
$ oc adm cordon <node_name>
Drain all pods on the node:
$ oc adm drain <node_name> --force=true
This step might fail if the node is offline or unresponsive. Even if the node does not respond, it might still be running a workload that writes to shared storage. To avoid data corruption, power down the physical hardware before you proceed.
Delete the node from the cluster:
$ oc delete node <node_name>
Although the node object is now deleted from the cluster, it can still rejoin the cluster after reboot or if the kubelet service is restarted. To permanently delete the node and all its data, you must decommission the node.
If you powered down the physical hardware, turn it back on so that the node can rejoin the cluster.