$ oc label node <node_name> node-role.kubernetes.io/worker-realtime=""
You can configure an OKD cluster to run real-time virtual machine (VM) workloads that require low and predictable latency. OKD provides the Node Tuning Operator to implement automatic tuning for real-time and low latency workloads.
You can configure an OKD cluster to run real-time workloads.
You have access to the cluster as a user with cluster-admin
permissions.
You have installed the OpenShift CLI (oc
).
You have installed the Node Tuning Operator.
Label a subset of the compute nodes with a custom role, for example, worker-realtime
:
$ oc label node <node_name> node-role.kubernetes.io/worker-realtime=""
You must use the default |
Create a new MachineConfigPool
manifest that contains the worker-realtime
label in the spec.machineConfigSelector
object:
MachineConfigPool
manifestapiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfigPool
metadata:
name: worker-realtime
labels:
machineconfiguration.openshift.io/role: worker-realtime
spec:
machineConfigSelector:
matchExpressions:
- key: machineconfiguration.openshift.io/role
operator: In
values:
- worker
- worker-realtime
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker-realtime: ""
You do not need to create a new |
If you created a new MachineConfigPool
manifest in step 2, apply it to the cluster by using the following command:
$ oc apply -f <real_time_mcp>.yaml
Create a PerformanceProfile
manifest that applies to the labeled nodes and the machine config pool that you created in the previous steps:
PerformanceProfile
manifestapiVersion: performance.openshift.io/v2
kind: PerformanceProfile
metadata:
name: profile-1
spec:
cpu:
isolated: 4-39,44-79
reserved: 0-3,40-43
globallyDisableIrqLoadBalancing: true
hugepages:
defaultHugepagesSize: 1G
pages:
- count: 8
size: 1G
realTimeKernel:
enabled: true
workloadHints:
highPowerConsumption: true
realTime: true
nodeSelector:
node-role.kubernetes.io/worker-realtime: ""
numa:
topologyPolicy: single-numa-node
Apply the PerformanceProfile
manifest:
$ oc apply -f <real_time_pp>.yaml
The compute nodes automatically reboot twice after you apply the |
Retrieve the name of the generated RuntimeClass
resource from the status.runtimeClass
field of the PerformanceProfile
object:
$ oc get performanceprofiles.performance.openshift.io profile-1 -o=jsonpath='{.status.runtimeClass}{"\n"}'
Set the previously obtained RuntimeClass
name as the default container runtime class for the virt-launcher
pods by editing the HyperConverged
custom resource (CR):
$ oc patch hyperconverged kubevirt-hyperconverged -n kubevirt-hyperconverged \
--type='json' -p='[{"op": "add", "path": "/spec/defaultRuntimeClass", "value":"<runtimeclass_name>"}]'
Editing the |
If your real-time-enabled compute nodes use simultaneous multithreading (SMT), enable the alignCPUs
feature gate by editing the HyperConverged
CR:
$ oc patch hyperconverged kubevirt-hyperconverged -n kubevirt-hyperconverged \
--type='json' -p='[{"op": "replace", "path": "/spec/featureGates/alignCPUs", "value": true}]'
Enabling |
You can configure a virtual machines (VM) to run real-time workloads.
Your cluster is configured to run real-time workloads.
You have installed the virtctl
tool.
Create a VirtualMachine
manifest to include information about CPU topology, CRI-O annotations, and huge pages:
VirtualMachine
manifestapiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
name: realtime-vm
spec:
template:
metadata:
annotations:
cpu-load-balancing.crio.io: disable (1)
cpu-quota.crio.io: disable (2)
irq-load-balancing.crio.io: disable (3)
spec:
domain:
cpu:
dedicatedCpuPlacement: true
isolateEmulatorThread: true
model: host-passthrough
numa:
guestMappingPassthrough: {}
realtime: {}
sockets: 1 (4)
cores: 4 (5)
threads: 1
devices:
autoattachGraphicsDevice: false
autoattachMemBalloon: false
autoattachSerialConsole: true
ioThreadsPolicy: auto
memory:
guest: 4Gi
hugepages:
pageSize: 1Gi (6)
terminationGracePeriodSeconds: 0
# ...
1 | This annotation specifies that load balancing is disabled for CPUs that are used by the container. |
2 | This annotation specifies that the CPU quota is disabled for CPUs that are used by the container. |
3 | This annotation specifies that interrupt request (IRQ) load balancing is disabled for CPUs that are used by the container. |
4 | The number of sockets inside the VM. |
5 | The number of cores inside the VM. This must be a value greater than or equal to 1 . |
6 | The size of the huge pages. The possible values for x86-64 architectures are 1Gi and 2Mi . In this example, the request is for 4 huge pages of size 1 Gi. |
Apply the VirtualMachine
manifest:
$ oc apply -f <file_name>.yaml
Configure the guest operating system. The following example shows the configuration steps for a Fedora 8 operating system:
Run the following command to connect to the VM console:
$ virtctl console <vm_name>
Configure huge pages by using the GRUB boot loader command-line interface. In the following example, 8 1G huge pages are specified.
$ grubby --update-kernel=ALL --args="default_hugepagesz=1GB hugepagesz=1G hugepages=8"
To achieve low-latency tuning by using the cpu-partitioning
profile in the TuneD application, run the following commands:
$ dnf install -y tuned-profiles-cpu-partitioning
$ echo isolated_cores=2-9 > /etc/tuned/cpu-partitioning-variables.conf
The first two CPUs (0 and 1) are set aside for house keeping tasks and the rest are isolated for the real-time application.
$ tuned-adm profile cpu-partitioning
Restart the VM to apply the changes.