×

Your single-node OpenShift clusters need to have a shared var/lib/containers partition for the image-based upgrade. You can do this at install time.

Configuring a shared container partition between ostree stateroots

Apply a MachineConfig to both the seed and the target clusters during installation time to create a separate partition and share the /var/lib/containers partition between the two ostree stateroots that will be used during the upgrade process.

You must complete this procedure at installation time.

Procedure
  • Apply a MachineConfig to create a separate partition:

    apiVersion: machineconfiguration.openshift.io/v1
    kind: MachineConfig
    metadata:
      labels:
        machineconfiguration.openshift.io/role: master
      name: 98-var-lib-containers-partitioned
    spec:
      config:
        ignition:
          version: 3.2.0
        storage:
          disks:
            - device: /dev/disk/by-path/pci-<root_disk> (1)
              partitions:
                - label: varlibcontainers
                  startMiB: <start_of_partition> (2)
                  sizeMiB: <partition_size> (3)
          filesystems:
            - device: /dev/disk/by-partlabel/varlibcontainers
              format: xfs
              mountOptions:
                - defaults
                - prjquota
              path: /var/lib/containers
              wipeFilesystem: true
        systemd:
          units:
            - contents: |-
                # Generated by Butane
                [Unit]
                Before=local-fs.target
                Requires=systemd-fsck@dev-disk-by\x2dpartlabel-varlibcontainers.service
                After=systemd-fsck@dev-disk-by\x2dpartlabel-varlibcontainers.service
    
                [Mount]
                Where=/var/lib/containers
                What=/dev/disk/by-partlabel/varlibcontainers
                Type=xfs
                Options=defaults,prjquota
    
                [Install]
                RequiredBy=local-fs.target
              enabled: true
              name: var-lib-containers.mount
    1 Specify the root disk.
    2 Specify the start of the partition in MiB. If the value is too small, the installation will fail.
    3 Specify a minimum size for the partition of 500 GB to ensure adequate disk space for precached images. If the value is too small, the deployments after installation will fail.

Configuring a shared container directory between ostree stateroots when using GitOps ZTP

When you are using the GitOps Zero Touch Provisioning (ZTP) workflow, you do the following procedure to create a separate disk partition on both the seed and target cluster and to share the /var/lib/containers partition.

You must complete this procedure at installation time.

Prerequisites
  • Install Butane. For more information, see "Installing Butane".

Procedure
  1. Create the storage.bu file:

    variant: fcos
    version: 1.3.0
    storage:
      disks:
      - device: /dev/disk/by-path/pci-<root_disk> (1)
        wipe_table: false
        partitions:
        - label: var-lib-containers
          start_mib: <start_of_partition> (2)
          size_mib: <partition_size> (3)
      filesystems:
        - path: /var/lib/containers
          device: /dev/disk/by-partlabel/var-lib-containers
          format: xfs
          wipe_filesystem: true
          with_mount_unit: true
          mount_options:
            - defaults
            - prjquota
    1 Specify the root disk.
    2 Specify the start of the partition in MiB. If the value is too small, the installation will fail.
    3 Specify a minimum size for the partition of 500 GB to ensure adequate disk space for precached images. If the value is too small, the deployments after installation will fail.
  2. Convert the storage.bu to an Ignition file by running the following command:

    $ butane storage.bu
    Example output
    {"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}
  3. Copy the output into the .spec.clusters.nodes.ignitionConfigOverride field in the SiteConfig CR:

    [...]
    spec:
      clusters:
        - nodes:
            - hostName: <name>
              ignitionConfigOverride: '{"ignition":{"version":"3.2.0"},"storage":{"disks":[{"device":"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0","partitions":[{"label":"var-lib-containers","sizeMiB":0,"startMiB":250000}],"wipeTable":false}],"filesystems":[{"device":"/dev/disk/by-partlabel/var-lib-containers","format":"xfs","mountOptions":["defaults","prjquota"],"path":"/var/lib/containers","wipeFilesystem":true}]},"systemd":{"units":[{"contents":"# Generated by Butane\n[Unit]\nRequires=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\nAfter=systemd-fsck@dev-disk-by\\x2dpartlabel-var\\x2dlib\\x2dcontainers.service\n\n[Mount]\nWhere=/var/lib/containers\nWhat=/dev/disk/by-partlabel/var-lib-containers\nType=xfs\nOptions=defaults,prjquota\n\n[Install]\nRequiredBy=local-fs.target","enabled":true,"name":"var-lib-containers.mount"}]}}'
    [...]
Verification
  1. During or after installation, verify on the hub cluster that the BareMetalHost object shows the annotation by running the following command:

    $ oc get bmh -n my-sno-ns my-sno -ojson | jq '.metadata.annotations["bmac.agent-install.openshift.io/ignition-config-overrides"]'
    Example output
    "{\"ignition\":{\"version\":\"3.2.0\"},\"storage\":{\"disks\":[{\"device\":\"/dev/disk/by-path/pci-0000:00:17.0-ata-1.0\",\"partitions\":[{\"label\":\"var-lib-containers\",\"sizeMiB\":0,\"startMiB\":250000}],\"wipeTable\":false}],\"filesystems\":[{\"device\":\"/dev/disk/by-partlabel/var-lib-containers\",\"format\":\"xfs\",\"mountOptions\":[\"defaults\",\"prjquota\"],\"path\":\"/var/lib/containers\",\"wipeFilesystem\":true}]},\"systemd\":{\"units\":[{\"contents\":\"# Generated by Butane\\n[Unit]\\nRequires=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\nAfter=systemd-fsck@dev-disk-by\\\\x2dpartlabel-var\\\\x2dlib\\\\x2dcontainers.service\\n\\n[Mount]\\nWhere=/var/lib/containers\\nWhat=/dev/disk/by-partlabel/var-lib-containers\\nType=xfs\\nOptions=defaults,prjquota\\n\\n[Install]\\nRequiredBy=local-fs.target\",\"enabled\":true,\"name\":\"var-lib-containers.mount\"}]}}"
  2. After installation, check the single-node OpenShift disk status by running the following commands:

    # lsblk
    Example output
    NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
    sda      8:0    0 446.6G  0 disk
    ├─sda1   8:1    0     1M  0 part
    ├─sda2   8:2    0   127M  0 part
    ├─sda3   8:3    0   384M  0 part /boot
    ├─sda4   8:4    0 243.6G  0 part /var
    │                                /sysroot/ostree/deploy/rhcos/var
    │                                /usr
    │                                /etc
    │                                /
    │                                /sysroot
    └─sda5   8:5    0 202.5G  0 part /var/lib/containers
    # df -h
    Example output
    Filesystem      Size  Used Avail Use% Mounted on
    devtmpfs        4.0M     0  4.0M   0% /dev
    tmpfs           126G   84K  126G   1% /dev/shm
    tmpfs            51G   93M   51G   1% /run
    /dev/sda4       244G  5.2G  239G   3% /sysroot
    tmpfs           126G  4.0K  126G   1% /tmp
    /dev/sda5       203G  119G   85G  59% /var/lib/containers
    /dev/sda3       350M  110M  218M  34% /boot
    tmpfs            26G     0   26G   0% /run/user/1000
Additional resources