1. Documentation
×

Updated boot images

The Machine Config Operator (MCO) uses a boot image to start a Fedora CoreOS (FCOS) node. By default, OKD does not manage the boot image.

This means that the boot image in your cluster is not updated along with your cluster. For example, if your cluster was originally created with OKD 4.12, the boot image that the cluster uses to create nodes is the same 4.12 version, even if your cluster is at a later version. If the cluster is later upgraded to 4.13 or later, new nodes continue to scale with the same 4.12 image.

This process could cause the following issues:

  • Extra time to start nodes

  • Certificate expiration issues

  • Version skew issues

To avoid these issues, you can configure your cluster to update the boot image whenever you update your cluster. By modifying the MachineConfiguration object, you can enable this feature. Currently, the ability to update the boot image is available for only Google Cloud Platform (GCP) and Amazon Web Services (AWS) clusters. It is not supported for clusters managed by the Cluster CAPI Operator.

If you are not using the default user data secret, named worker-user-data, in your machine set, or you have modified the worker-user-data secret, you should not use managed boot image updates. This is because the Machine Config Operator (MCO) updates the machine set to use a managed version of the secret. By using the managed boot images feature, you are giving up the capability to customize the secret stored in the machine set object.

To view the current boot image used in your cluster, examine a machine set:

Example machine set with the boot image reference
apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  name: ci-ln-hmy310k-72292-5f87z-worker-a
  namespace: openshift-machine-api
spec:
# ...
  template:
# ...
    spec:
# ...
      providerSpec:
# ...
        value:
          disks:
          - autoDelete: true
            boot: true
            image: projects/rhcos-cloud/global/images/rhcos-412-85-202203181601-0-gcp-x86-64 (1)
# ...
1 This boot image is the same as the originally-installed OKD version, in this example OKD 4.12, regardless of the current version of the cluster. The way that the boot image is represented in the machine set depends on the platform, as the structure of the providerSpec field differs from platform to platform.

If you configure your cluster to update your boot images, the boot image referenced in your machine sets matches the current version of the cluster.

Configuring updated boot images

By default, OKD does not manage the boot image. You can configure your cluster to update the boot image whenever you update your cluster by modifying the MachineConfiguration object.

Currently, the ability to update the boot image is available for only Google Cloud Platform (GCP) clusters and as a Technology Preview feature for Amazon Web Services (AWS) clusters. It is not supported for clusters managed by the Cluster CAPI Operator.

The updating boot image feature for AWS is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.
Procedure

  1. Edit the MachineConfiguration object, named cluster, to enable the updating of boot images by running the following command:

    $ oc edit MachineConfiguration cluster

    • Optional: Configure the boot image update feature for all the machine sets:

      apiVersion: operator.openshift.io/v1
kind: MachineConfiguration
metadata:
  name: cluster
  namespace: openshift-machine-config-operator
spec:
# ...
  managedBootImages: (1)
    machineManagers:
    - resource: machinesets
      apiGroup: machine.openshift.io
      selection:
        mode: All (2)
      1 Activates the boot image update feature.
      2 Specifies that all the machine sets in the cluster are to be updated.

    • Optional: Configure the boot image update feature for specific machine sets:

      apiVersion: operator.openshift.io/v1
kind: MachineConfiguration
metadata:
  name: cluster
  namespace: openshift-machine-config-operator
spec:
# ...
  managedBootImages: (1)
    machineManagers:
    - resource: machinesets
      apiGroup: machine.openshift.io
      selection:
        mode: Partial
        partial:
          machineResourceSelector:
            matchLabels:
              update-boot-image: "true" (2)
      1 Activates the boot image update feature.
      2 Specifies that any machine set with this label is to be updated.

      If an appropriate label is not present on the machine set, add a key/value pair by running a command similar to following:

      $ oc label machineset.machine ci-ln-hmy310k-72292-5f87z-worker-a update-boot-image=true -n openshift-machine-api
Verification

  1. View the current state of the boot image updates by viewing the machine configuration object:

    $ oc get machineconfiguration cluster -n openshift-machine-api -o yaml
    Example machine set with the boot image reference
    kind: MachineConfiguration
metadata:
  name: cluster
# ...
status:
  conditions:
  - lastTransitionTime: "2024-09-09T13:51:37Z" (1)
    message: Reconciled 1 of 2 MAPI MachineSets | Reconciled 0 of 0 CAPI MachineSets
      | Reconciled 0 of 0 CAPI MachineDeployments
    reason: BootImageUpdateConfigurationAdded
    status: "True"
    type: BootImageUpdateProgressing
  - lastTransitionTime: "2024-09-09T13:51:37Z" (2)
    message: 0 Degraded MAPI MachineSets | 0 Degraded CAPI MachineSets | 0 CAPI MachineDeployments
    reason: BootImageUpdateConfigurationAdded
    status: "False"
    type: BootImageUpdateDegraded
    1 Status of the boot image update. Cluster CAPI Operator machine sets and machine deployments are not currently supported for boot image updates.
    2 Indicates if any boot image updates failed. If any of the updates fail, the Machine Config Operator is degraded. In this case, manual intervention might be required.

  2. Get the boot image version by running the following command:

    $ oc get machinesets <machineset_name> -n openshift-machine-api -o yaml
    Example machine set with the boot image reference
    apiVersion: machine.openshift.io/v1beta1
kind: MachineSet
metadata:
  labels:
    machine.openshift.io/cluster-api-cluster: ci-ln-77hmkpt-72292-d4pxp
    update-boot-image: "true"
  name: ci-ln-77hmkpt-72292-d4pxp-worker-a
  namespace: openshift-machine-api
spec:
# ...
  template:
# ...
    spec:
# ...
      providerSpec:
# ...
        value:
          disks:
          - autoDelete: true
            boot: true
            image: projects/rhcos-cloud/global/images/rhcos-416-92-202402201450-0-gcp-x86-64 (1)
# ...
    1 This boot image is the same as the current OKD version.
Additional resources

Disabling updated boot images

To disable the updated boot image feature, edit the MachineConfiguration object to remove the managedBootImages stanza.

If you disable this feature after some nodes have been created with the new boot image version, any existing nodes retain their current boot image. Turning off this feature does not rollback the nodes or machine sets to the originally-installed boot image. The machine sets retain the boot image version that was present when the feature was enabled and is not updated again when the cluster is upgraded to a new OKD version in the future.

Procedure

  1. Disable updated boot images by editing the MachineConfiguration object:

    $ oc edit MachineConfiguration cluster

  2. Remove the managedBootImages stanza:

    apiVersion: operator.openshift.io/v1
kind: MachineConfiguration
metadata:
  name: cluster
  namespace: openshift-machine-config-operator
spec:
# ...
  managedBootImages: (1)
    machineManagers:
    - resource: machinesets
      apiGroup: machine.openshift.io
      selection:
        mode: All
    1 Remove the entire stanza to disable updated boot images.