1. Documentation
×

Persistent storage using Logical Volume Manager Storage

Logical Volume Manager (LVM) Storage uses Logical Volume Manager (LVM2) through the TopoLVM Container Storage Interface (CSI) driver to dynamically provision local storage on a cluster with limited resources.

You can create volume groups, persistent volume claims (PVCs), volume snapshots, and volume clones by using LVM Storage.

Logical Volume Manager Storage installation

You can install Logical Volume Manager (LVM) Storage on a single-node OpenShift cluster and configure it to dynamically provision storage for your workloads.

You can deploy LVM Storage on single-node OpenShift clusters by using the OKD CLI (oc), OKD web console, or Red Hat Advanced Cluster Management (RHACM).

Prerequisites to install LVM Storage

The prerequisites to install LVM Storage are as follows:

  • Ensure that you have a minimum of 10 milliCPU and 100 MiB of RAM.

  • Ensure that every managed cluster has dedicated disks that are used to provision storage. LVM Storage uses only those disks that are empty and do not contain file system signatures. To ensure that the disks are empty and do not contain file system signatures, wipe the disks before using them.

  • Before installing LVM Storage in a private CI environment where you can reuse the storage devices that you configured in the previous LVM Storage installation, ensure that you have wiped the disks that are not in use. If you do not wipe the disks before installing LVM Storage, you cannot reuse the disks without manual intervention.

    You cannot wipe the disks that are in use.

  • If you want to install LVM Storage by using Red Hat Advanced Cluster Management (RHACM), ensure that you have installed RHACM on an OKD cluster. See the Installing LVM Storage using RHACM section.

Additional resources

Installing LVM Storage by using the CLI

As a cluster administrator, you can install Logical Volume Manager (LVM) Storage by using the OpenShift CLI (oc).

Prerequisites

  • You have installed the OpenShift CLI (oc).

  • You have logged in to OKD as a user with cluster-admin and Operator installation permissions.

Procedure

  1. Create a YAML file and add the configuration for creating a namespace.

    Example YAML configuration for creating a namespace
    apiVersion: v1
kind: Namespace
metadata:
  labels:
    openshift.io/cluster-monitoring: "true"
    pod-security.kubernetes.io/enforce: privileged
    pod-security.kubernetes.io/audit: privileged
    pod-security.kubernetes.io/warn: privileged
  name: openshift-storage

  2. Create the namespace by running the following command:

    $ oc create -f <file_name>

  3. Create an OperatorGroup custom resource (CR) YAML file.

    Example OperatorGroup CR
    apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
  name: openshift-storage-operatorgroup
  namespace: openshift-storage
spec:
  targetNamespaces:
  - openshift-storage

  4. Create the OperatorGroup CR by running the following command:

    $ oc create -f <file_name>

  5. Create a Subscription CR YAML file.

    Example Subscription CR
    apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
  name: lvms
  namespace: openshift-storage
spec:
  installPlanApproval: Automatic
  name: lvms-operator
  source: redhat-operators
  sourceNamespace: openshift-marketplace

  6. Create the Subscription CR by running the following command:

    $ oc create -f <file_name>
Verification

  1. To verify that LVM Storage is installed, run the following command:

    $ oc get csv -n openshift-storage -o custom-columns=Name:.metadata.name,Phase:.status.phase
    Example output
    Name                         Phase
4.13.0-202301261535          Succeeded

Installing LVM Storage by using the web console

You can install Logical Volume Manager (LVM) Storage by using the OKD web console.

Prerequisites

  • You have access to the single-node OpenShift cluster.

  • You have access to OKD with cluster-admin and Operator installation permissions.

Procedure

  1. Log in to the OKD web console.

  2. Click Operators → OperatorHub.

  3. Click LVM Storage on the OperatorHub page.

  4. Set the following options on the Operator Installation page:

    1. Update Channel as stable-4.14.

    2. Installation Mode as A specific namespace on the cluster.

    3. Installed Namespace as Operator recommended namespace openshift-storage. If the openshift-storage namespace does not exist, it is created during the operator installation.

    4. Update approval as Automatic or Manual.

      If you select Automatic updates, the Operator Lifecycle Manager (OLM) automatically updates the running instance of LVM Storage without any intervention.

      If you select Manual updates, the OLM creates an update request. As a cluster administrator, you must manually approve the update request to update LVM Storage to a newer version.

  5. Optional: Select the Enable Operator recommended cluster monitoring on this Namespace checkbox.

  6. Click Install.

Verification steps

  • Verify that LVM Storage shows a green tick, indicating successful installation.

Installing LVM Storage in a disconnected environment

You can install Logical Volume Manager (LVM) Storage on OKD 4.14 in a disconnected environment. All sections referenced in this procedure are linked in the "Additional resources" section.

Prerequisites

  • You read the "About disconnected installation mirroring" section.

  • You have access to the OKD image repository.

  • You created a mirror registry.

Procedure

  1. Follow the steps in the "Creating the image set configuration" procedure. To create an image set configuration for LVM Storage, you can use the following example ImageSetConfiguration object configuration:

    Example ImageSetConfiguration file for LVM Storage
    kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
archiveSize: 4 (1)
storageConfig: (2)
  registry:
    imageURL: example.com/mirror/oc-mirror-metadata (3)
    skipTLS: false
mirror:
  platform:
    channels:
    - name: stable-4.14 (4)
      type: ocp
    graph: true (5)
  operators:
  - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14 (6)
    packages:
    - name: lvms-operator (7)
      channels:
      - name: stable (8)
  additionalImages:
  - name: registry.redhat.io/ubi9/ubi:latest (9)
  helm: {}
    1 Set the the maximum size (in gibibytes) of each file within the image set.
    2 Specify the location in which you want to save the image set. This location can be a registry or a local directory.
    3 Specify the storage URL for the image stream when using a registry. For more information, see "Why use imagestreams".
    4 Specify the channel from which you want to retrieve the OKD images.
    5 Set this field to true to generate the OpenShift Update Service (OSUS) graph image. For more information, see "About the OpenShift Update Service".
    6 Specify the Operator catalog from which you want to retrieve the OKD images.
    7 Specify the Operator packages to include in the image set. If this field is empty, all packages in the catalog are retrieved.
    8 Specify the channels of the Operator packages to include in the image set. You must include the default channel for the Operator package even if you do not use the bundles in that channel. You can find the default channel by running the following command: $ oc mirror list operators --catalog=<catalog_name> --package=<package_name>.
    9 Specify any additional images to include in the image set.

  2. Follow the procedure in the "Mirroring an image set to a mirror registry" section.

  3. Follow the procedure in the "Configuring image registry repository mirroring" section.

Additional resources

Installing LVM Storage using RHACM

LVM Storage is deployed on single-node OpenShift clusters using Red Hat Advanced Cluster Management (RHACM). You create a Policy object on RHACM that deploys and configures the Operator when it is applied to managed clusters which match the selector specified in the PlacementRule resource. The policy is also applied to clusters that are imported later and satisfy the placement rule.

Prerequisites

  • Access to the RHACM cluster using an account with cluster-admin and Operator installation permissions.

  • Dedicated disks on each single-node OpenShift cluster to be used by LVM Storage.

  • The single-node OpenShift cluster needs to be managed by RHACM, either imported or created.

Procedure

  1. Log in to the RHACM CLI using your OKD credentials.

  2. Create a namespace in which you will create policies.

    # oc create ns lvms-policy-ns

  3. To create a policy, save the following YAML to a file with a name such as policy-lvms-operator.yaml:

    apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-install-lvms
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector: (1)
    matchExpressions:
    - key: mykey
      operator: In
      values:
      - myvalue
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-install-lvms
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-install-lvms
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: install-lvms
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: install-lvms
spec:
  disabled: false
  remediationAction: enforce
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: install-lvms
      spec:
        object-templates:
        - complianceType: musthave
          objectDefinition:
            apiVersion: v1
            kind: Namespace
            metadata:
              labels:
                openshift.io/cluster-monitoring: "true"
                pod-security.kubernetes.io/enforce: privileged
                pod-security.kubernetes.io/audit: privileged
                pod-security.kubernetes.io/warn: privileged
              name: openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1
            kind: OperatorGroup
            metadata:
              name: openshift-storage-operatorgroup
              namespace: openshift-storage
            spec:
              targetNamespaces:
              - openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1alpha1
            kind: Subscription
            metadata:
              name: lvms
              namespace: openshift-storage
            spec:
              installPlanApproval: Automatic
              name: lvms-operator
              source: redhat-operators
              sourceNamespace: openshift-marketplace
        remediationAction: enforce
        severity: low
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: lvms
      spec:
        object-templates:
           - complianceType: musthave
             objectDefinition:
               apiVersion: lvm.topolvm.io/v1alpha1
               kind: LVMCluster
               metadata:
                 name: my-lvmcluster
                 namespace: openshift-storage
               spec:
                 storage:
                   deviceClasses:
                   - name: vg1
                     default: true
                     deviceSelector: (2)
                       paths:
                       - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
                       - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
                       optionalPaths:
                       - /dev/disk/by-path/pci-0000:89:00.0-nvme-1
                       - /dev/disk/by-path/pci-0000:90:00.0-nvme-1
                     thinPoolConfig:
                       name: thin-pool-1
                       sizePercent: 90
                       overprovisionRatio: 10
                     nodeSelector: (3)
                       nodeSelectorTerms:
                       - matchExpressions:
                           - key: app
                             operator: In
                             values:
                             - test1
        remediationAction: enforce
        severity: low
    1 Replace the key and value in PlacementRule.spec.clusterSelector to match the labels set on the single-node OpenShift clusters on which you want to install LVM Storage.
    2 Optional. To control or restrict the volume group to your preferred devices, you can manually specify the local paths of the devices in the deviceSelector section of the LVMCluster YAML. The paths section refers to devices the LVMCluster adds, which means those paths must exist. The optionalPaths section refers to devices the LVMCluster might add. You must specify at least one of paths or optionalPaths when specifying the deviceSelector section. If you specify paths, it is not mandatory to specify optionalPaths. If you specify optionalPaths, it is not mandatory to specify paths but at least one optional path must be present on the node. If you do not specify any paths, it will add all unused devices on the node.
    3 To add a node filter, which is a subset of the additional worker nodes, specify the required filter in the nodeSelector section. LVM Storage detects and uses the additional worker nodes when the new nodes show up.

    This nodeSelector node filter matching is not the same as the pod label matching.

  4. Create the policy in the namespace by running the following command:

    # oc create -f policy-lvms-operator.yaml -n lvms-policy-ns (1)
    1 The policy-lvms-operator.yaml is the name of the file to which the policy is saved.

    This creates a Policy, a PlacementRule, and a PlacementBinding object in the lvms-policy-ns namespace. The policy creates a Namespace, OperatorGroup, Subscription, and LVMCluster resource on the clusters that match the placement rule. This deploys the Operator on the single-node OpenShift clusters which match the selection criteria and configures it to set up the required resources to provision storage. The Operator uses all the disks specified in the LVMCluster CR. If no disks are specified, the Operator uses all the unused disks on the single-node OpenShift node.

    After a device is added to the LVMCluster, it cannot be removed.
Additional resources

Limitations to configure the size of the devices used in LVM Storage

The limitations to configure the size of the devices that you can use to provision storage using LVM Storage are as follows:

  • The total storage size that you can provision is limited by the size of the underlying Logical Volume Manager (LVM) thin pool and the over-provisioning factor.

  • The size of the logical volume depends on the size of the Physical Extent (PE) and the Logical Extent (LE).

    • You can define the size of PE and LE during the physical and logical device creation.

    • The default PE and LE size is 4 MB.

    • If the size of the PE is increased, the maximum size of the LVM is determined by the kernel limits and your disk space.

Table 1. Size limits for different architectures using the default PE and LE size
Architecture RHEL 6 RHEL 7 RHEL 8 RHEL 9

32-bit

16 TB

-

-

-

64-bit

8 EB [1]

100 TB [2]

8 EB [1]

500 TB [2]

8 EB

8 EB

  1. Theoretical size.

  2. Tested size.

About the LVMCluster custom resource

You can configure the LVMCluster custom resource (CR) to perform the following actions:

  • Create LVM volume groups that you can use to provision persistent volume claims (PVCs).

  • Configure a list of devices that you want to add to the LVM volume groups.

  • Configure the requirements to select the nodes on which you want to create an LVM volume group, and the thin pool configuration for the volume group.

After you have installed LVM Storage, you must create an LVMCluster custom resource (CR).

Example LVMCluster CR YAML file
apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
  name: my-lvmcluster
spec:
  tolerations:
  - effect: NoSchedule
    key: xyz
    operator: Equal
    value: "true"
  storage:
    deviceClasses:
    - name: vg1
      fstype: ext4 (1)
      default: true
      nodeSelector: (1)
        nodeSelectorTerms:
        - matchExpressions:
          - key: mykey
            operator: In
            values:
            - ssd
      deviceSelector: (1)
        paths:
        - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
        optionalPaths:
        - /dev/disk/by-path/pci-0000:89:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:90:00.0-nvme-1
      thinPoolConfig:
        name: thin-pool-1
        sizePercent: 90 (1)
        overprovisionRatio: 10
1 Optional field

Explanation of fields in the LVMCluster CR

The LVMCluster CR fields are described in the following table:

Table 2. LVMCluster CR fields
Field Type Description

spec.storage.deviceClasses

array

Contains the configuration to assign the local storage devices to the LVM volume groups.

LVM Storage creates a storage class and volume snapshot class for each device class that you create.

If you add or remove a device class, the update reflects in the cluster only after deleting and recreating the topolvm-node pod.

deviceClasses.name

string

Specify a name for the LVM volume group (VG).

deviceClasses.fstype

string

Set this field to ext4 or xfs. By default, this field is set to xfs.

deviceClasses.default

boolean

Set this field to true to indicate that a device class is the default. Otherwise, you can set it to false. You can only configure a single default device class.

deviceClasses.nodeSelector

object

Contains the configuration to choose the nodes on which you want to create the LVM volume group. If this field is empty, all nodes without no-schedule taints are considered.

On the control-plane node, LVM Storage detects and uses the additional worker nodes when the new nodes become active in the cluster.

nodeSelector.nodeSelectorTerms

array

Configure the requirements that are used to select the node.

deviceClasses.deviceSelector

object

Contains the configuration to specify the paths to the devices that you want to add to the LVM volume group.

For more information, see "About adding devices to a volume group".

deviceSelector.paths

array

Specify the device paths.

If the device path specified in this field does not exist, the LVMCluster CR moves to the Failed state.

deviceSelector.optionalPaths

array

Specify the optional device paths.

If the device path specified in this field does not exist, LVM Storage ignores the device without causing an error.

deviceClasses.thinPoolConfig

object

Contains the configuration to create a thin pool in the LVM volume group.

thinPoolConfig.name

string

Specify a name for the thin pool.

thinPoolConfig.sizePercent

integer

Specify the percentage of space in the LVM volume group for creating the thin pool.

By default, this field is set to 90. The minimum value that you can set is 10, and the maximum value is 90.

thinPoolConfig.overprovisionRatio

integer

Specify a factor by which you can provision additional storage based on the available storage in the thin pool.

For example, if this field is set to 10, you can provision up to 10 times the amount of available storage in the thin pool.

To disable over-provisioning, set this field to 1.
Additional resources

About adding devices to a volume group

The deviceSelector field in the LVMCluster custom resource (CR) contains the configuration to specify the paths to the devices that you want to add to the LVM volume group.

You can specify the device paths in the deviceSelector.paths field, the deviceSelector.optionalPaths field, or both. If you do not specify the device paths in both the deviceSelector.paths field and the deviceSelector.optionalPaths field, LVM Storage adds the unused devices to the LVM volume group.

If you do not add the deviceSelector field in the LVMCluster CR, LVM Storage automatically adds the new devices when the devices are available.

LVM Storage adds the devices to the LVM volume group only if the device path exists.

After a device is added to the LVM volume group, it cannot be removed.

Ways to create an LVMCluster custom resource

You can create an LVMCluster custom resource (CR) by using the OpenShift CLI (oc) or the OKD web console. If you have installed LVM Storage by using Red Hat Advanced Cluster Management (RHACM), you can also create an LVMCluster CR by using RHACM.

Upon creating the LVMCluster CR, LVM Storage creates the following system-managed CRs:

  • A storageClass and volumeSnapshotClass for each device class.

    LVM Storage configures the name of the storage class and volume snapshot class in the format lvms-<device_class_name>, where, <device_class_name> is the value of the deviceClasses.name field in the LVMCluster CR. For example, if the deviceClasses.name field is set to vg1, the name of the storage class and volume snapshot class is lvms-vg1.

  • LVMVolumeGroup: This CR is a specific type of persistent volume (PV) that is backed by an LVM volume group. It tracks the individual volume groups across multiple nodes.

  • LVMVolumeGroupNodeStatus: This CR tracks the status of the volume groups on a node.

Creating an LVMCluster CR by using the CLI

You can create an LVMCluster custom resource (CR) on a worker node using the OpenShift CLI (oc).

You can only create a single instance of the LVMCluster custom resource (CR) on an OKD cluster.
Prerequisites

  • You have installed the OpenShift CLI (oc).

  • You have logged in to OKD as a user with cluster-admin privileges.

  • You have installed LVM Storage.

  • You have installed a worker node in the cluster.

  • You read the "About the LVMCluster custom resource" section.

Procedure

  1. Create an LVMCluster custom resource (CR) YAML file:

    Example LVMCluster CR YAML file
    apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
  name: my-lvmcluster
spec:
# ...
  storage:
    deviceClasses: (1)
# ...
      nodeSelector: (2)
# ...
      deviceSelector: (3)
# ...
      thinPoolConfig: (4)
# ...
    1 Contains the configuration to assign the local storage devices to the LVM volume groups.
    2 Contains the configuration to choose the nodes on which you want to create the LVM volume group. If this field is empty, all nodes without no-schedule taints are considered.
    3 Contains the configuration to specify the paths to the devices that you want to add to the LVM volume group.
    4 Contains the configuration to create a thin pool in the LVM volume group.

  2. Create the LVMCluster CR by running the following command:

    $ oc create -f <file_name>
    Example output
    lvmcluster/lvmcluster created
Verification

  1. Check that the LVMCluster CR is in the Ready state:

    $ oc get lvmclusters.lvm.topolvm.io -o jsonpath='{.items[*].status.state}' -n <namespace>
    Example output
    {"deviceClassStatuses": (1)
[
  {
    "name": "vg1",
    "nodeStatus": [ (2)
        {
            "devices": [ (3)
                "/dev/nvme0n1",
                "/dev/nvme1n1",
                "/dev/nvme2n1"
            ],
            "node": "kube-node", (4)
            "status": "Ready" (5)
        }
    ]
  }
]
"state":"Ready"} (6)
    1 The status of the device class.
    2 The status of the LVM volume group on each node.
    3 The list of devices used to create the LVM volume group.
    4 The node on which the device class is created.
    5 The status of the LVM volume group on the node.
    6 The status of the LVMCluster CR.

    If the LVMCluster CR is in the Failed state, you can view the reason for failure in the status field.

    Example status field with the reason for failure:

    status:
  deviceClassStatuses:
    - name: vg1
      nodeStatus:
        - node: my-node-1.example.com
          reason: no available devices found for volume group
          status: Failed
  state: Failed

  2. Optional: To view the storage classes created by LVM Storage for each device class, run the following command:

    $ oc get storageclass
    Example output
    NAME          PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
lvms-vg1      topolvm.io           Delete          WaitForFirstConsumer   true                   31m

  3. Optional: To view the volume snapshot classes created by LVM Storage for each device class, run the following command:

    $ oc get volumesnapshotclass
    Example output
    NAME          DRIVER               DELETIONPOLICY   AGE
lvms-vg1      topolvm.io           Delete           24h
Additional resources

Creating an LVMCluster CR by using the web console

You can create an LVMCluster CR on a worker node using the OKD web console.

You can only create a single instance of the LVMCluster custom resource (CR) on an OKD cluster.
Prerequisites

  • You have access to the OKD cluster with cluster-admin privileges.

  • You have installed LVM Storage.

  • You have installed a worker node in the cluster.

  • You read the "About the LVMCluster custom resource" section.

Procedure

  1. Log in to the OKD web console.

  2. Click Operators → Installed Operators.

  3. In the openshift-storage namespace, click LVM Storage.

  4. Click Create LVMCluster and select either Form view or YAML view.

  5. Configure the required LVMCluster CR parameters.

  6. Click Create.

  7. Optional: If you want to edit the LVMCLuster CR, perform the following actions:

    1. Click the LVMCluster tab.

    2. From the Actions menu, select Edit LVMCluster.

    3. Click YAML and edit the required LVMCLuster CR parameters.

    4. Click Save.

Verification

  1. On the LVMCLuster page, check that the LVMCluster CR is in the Ready state.

  2. Optional: To view the available storage classes created by LVM Storage for each device class, click Storage → StorageClasses.

  3. Optional: To view the available volume snapshot classes created by LVM Storage for each device class, click Storage → VolumeSnapshotClasses.

Additional resources

Ways to delete an LVMCluster custom resource

You can delete an LVMCluster custom resource (CR) by using the OpenShift CLI (oc) or the OKD web console. If you have installed LVM Storage by using Red Hat Advanced Cluster Management (RHACM), you can also delete an LVMCluster CR by using RHACM.

Upon deleting the LVMCluster CR, LVM Storage deletes the following CRs:

  • storageClass

  • volumeSnapshotClass

  • LVMVolumeGroup

  • LVMVolumeGroupNodeStatus

Deleting an LVMCluster CR by using the CLI

You can delete the LVMCluster custom resource (CR) using the OpenShift CLI (oc).

Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

  • You have deleted the persistent volume claims (PVCs), volume snapshots, and volume clones provisioned by LVM Storage. You have also deleted the applications that are using these resources.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Delete the LVMCluster CR by running the following command:

    $ oc delete lvmcluster <lvmclustername> -n openshift-storage
Verification

  • To verify that the LVMCluster CR has been deleted, run the following command:

    $ oc get lvmcluster -n <namespace>
    Example output
    No resources found in openshift-storage namespace.

Deleting an LVMCluster CR by using the web console

You can delete the LVMCluster custom resource (CR) using the OKD web console.

Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

  • You have deleted the persistent volume claims (PVCs), volume snapshots, and volume clones provisioned by LVM Storage. You have also deleted the applications that are using these resources.

Procedure

  1. Log in to the OKD web console.

  2. Click Operators → Installed Operators to view all the installed Operators.

  3. Click LVM Storage in the openshift-storage namespace.

  4. Click the LVMCluster tab.

  5. From the Actions, select Delete LVMCluster.

  6. Click Delete.

Verification

  • On the LVMCLuster page, check that the LVMCluster CR has been deleted.

Adding a storage class

You can add a storage class to an OKD cluster. A storage class describes a class of storage in the cluster and how the cluster dynamically provisions the persistent volumes (PVs) when the user specifies the storage class. A storage class describes the type of device classes, the quality-of-service level, the filesystem type, and other details.

Procedure

  1. Create a YAML file:

    apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: lvm-storageclass
parameters:
  csi.storage.k8s.io/fstype: ext4
  topolvm.io/device-class: vg1
provisioner: topolvm.io
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer

    Save the file by using a name similar to the storage class name. For example, lvm-storageclass.yaml.

  2. Apply the YAML file by using the oc command:

    $ oc apply -f <file_name> (1)
    1 Replace <file_name> with the name of the YAML file. For example, lvm-storageclass.yaml.

    The cluster will create the storage class.

  3. Verify that the cluster created the storage class by using the following command:

    $ oc get storageclass <name> (1)
    1 Replace <name> with the name of the storage class. For example, lvm-storageclass.
    Example output
    NAME              PROVISIONER  RECLAIMPOLICY  VOLUMEBINDINGMODE     ALLOWVOLUMEEXPANSION  AGE
lvm-storageclass  topolvm.io   Delete         WaitForFirstConsumer  true                  1s

Provisioning storage using LVM Storage

You can provision persistent volume claims (PVCs) using the storage class that is created during the Operator installation. You can provision block and file PVCs, however, the storage is allocated only when a pod that uses the PVC is created.

LVM Storage provisions PVCs in units of 1 GiB. The requested storage is rounded up to the nearest GiB.
Procedure

  1. Identify the StorageClass that is created when LVM Storage is deployed.

    The StorageClass name is in the format, lvms-<device-class-name>. The device-class-name is the name of the device class that you provided in the LVMCluster of the Policy YAML. For example, if the deviceClass is called vg1, then the storageClass name is lvms-vg1.

    The volumeBindingMode of the storage class is set to WaitForFirstConsumer.

  2. To create a PVC where the application requires storage, save the following YAML to a file with a name such as pvc.yaml.

    Example YAML to create a PVC
    # block pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lvm-block-1
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Block
  resources:
    requests:
      storage: 10Gi
  storageClassName: lvms-vg1
---
# file pvc
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: lvm-file-1
  namespace: default
spec:
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 10Gi
  storageClassName: lvms-vg1

  3. Create the PVC by running the following command:

    # oc create -f pvc.yaml -ns <application_namespace>

    The created PVCs remain in pending state until you deploy the pods that use them.

Ways to scale up the storage of a single-node OpenShift cluster

You can scale up the storage of a single-node OpenShift cluster by adding new devices to the existing node.

To add a new device to the existing node on a single-node OpenShift cluster, you must add the path to the new device in the deviceSelector field of the LVMCluster custom resource (CR).

You can add the deviceSelector field in the LVMCluster CR only while creating the LVMCluster CR. If you have not added the deviceSelector field while creating the LVMCluster CR, you must delete the LVMCluster CR and create a new LVMCluster CR containing the deviceSelector field.

If you do not add the deviceSelector field in the LVMCluster CR, LVM Storage automatically adds the new devices when the devices are available.

Additional resources

Scaling up the storage of a single-node OpenShift cluster by using the CLI

You can scale up the storage capacity of the existing node on a single-node OpenShift cluster by using the OpenShift CLI (oc).

Prerequisites

  • You have additional unused devices on the single-node OpenShift cluster to be used by Logical Volume Manager (LVM) Storage.

  • You have installed the OpenShift CLI (oc).

  • You have created an LVMCluster custom resource (CR).

Procedure

  1. Edit the LVMCluster CR by running the following command:

    $ oc edit <lvmcluster_file_name> -n <namespace>

  2. Add the path to the new device in the deviceSelector field:

    Example LVMCluster CR
    apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
  name: my-lvmcluster
spec:
  storage:
    deviceClasses:
# ...
      deviceSelector: (1)
        paths: (2)
        - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
        optionalPaths: (3)
        - /dev/disk/by-path/pci-0000:89:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:90:00.0-nvme-1
# ...
    1 Contains the configuration to specify the paths to the devices that you want to add to the Logical Volume Manager (LVM) volume group. You can specify the device paths in the paths field, the optionalPaths field, or both. If you do not specify the device paths in both paths and optionalPaths, LVM Storage adds the supported unused devices to the LVM volume group. LVM Storage adds the devices to the LVM volume group only if the device path exists.
    2 Specify the device paths. If the device path specified in this field does not exist, the LVMCluster CR moves to the Failed state.
    3 Specify the optional device paths. If the device path specified in this field does not exist, LVM Storage ignores the device without causing an error.

    After a device is added to the LVM volume group, it cannot be removed.

  3. Save the LVMCluster CR.

Additional resources

Scaling up the storage of a single-node OpenShift cluster by using the web console

You can scale up the storage capacity of the existing node on a single-node OpenShift cluster by using the OKD web console.

Prerequisites

  • You have additional unused devices on the single-node OpenShift cluster to be used by Logical Volume Manager (LVM) Storage.

  • You have created an LVMCluster custom resource (CR).

Procedure

  1. Log in to the OKD web console.

  2. Click Operators → Installed Operators.

  3. Click LVM Storage in the openshift-storage namespace.

  4. Click the LVMCluster tab to view the LVMCluster CR created on the cluster.

  5. From the Actions menu, select Edit LVMCluster.

  6. Click the YAML tab.

  7. Edit the LVMCluster CR to add the new device path in the deviceSelector field:

    Example LVMCluster CR
    apiVersion: lvm.topolvm.io/v1alpha1
kind: LVMCluster
metadata:
  name: my-lvmcluster
spec:
  storage:
    deviceClasses:
# ...
      deviceSelector: (1)
        paths: (2)
        - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
        optionalPaths: (3)
        - /dev/disk/by-path/pci-0000:89:00.0-nvme-1
        - /dev/disk/by-path/pci-0000:90:00.0-nvme-1
# ...
    1 Contains the configuration to specify the paths to the devices that you want to add to the Logical Volume Manager (LVM) volume group. You can specify the device paths in the paths field, the optionalPaths field, or both. If you do not specify the device paths in both paths and optionalPaths, LVM Storage adds the supported unused devices to the LVM volume group. LVM Storage adds the devices to the LVM volume group only if the device path exists.
    2 Specify the device paths. If the device path specified in this field does not exist, the LVMCluster CR moves to the Failed state.
    3 Specify the optional device paths. If the device path specified in this field does not exist, LVM Storage ignores the device without causing an error.

    After a device is added to the LVM volume group, it cannot be removed.

  8. Click Save.

Additional resources

Scaling up storage by adding capacity to your single-node OpenShift cluster using RHACM

You can scale the storage capacity of your configured worker nodes on a single-node OpenShift cluster using RHACM.

Prerequisites

  • You have access to the RHACM cluster using an account with cluster-admin privileges.

  • You have additional unused devices on each single-node OpenShift cluster that LVM Storage can use.

Procedure

  1. Log in to the RHACM CLI using your OKD credentials.

  2. Find the device that you want to add. The device to be added needs to match with the device name and path of the existing devices.

  3. To add capacity to the single-node OpenShift cluster, edit the deviceSelector section of the existing policy YAML, for example, policy-lvms-operator.yaml.

    In case the deviceSelector field is not included during the LVMCluster creation, it is not possible to add the deviceSelector section to the CR. You need to remove the LVMCluster and then recreate it from the new CR.

    		 
    apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-install-lvms
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
    - key: mykey
      operator: In
      values:
      - myvalue
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-install-lvms
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-install-lvms
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: install-lvms
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: install-lvms
spec:
  disabled: false
  remediationAction: enforce
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: install-lvms
      spec:
        object-templates:
        - complianceType: musthave
          objectDefinition:
            apiVersion: v1
            kind: Namespace
            metadata:
              labels:
                openshift.io/cluster-monitoring: "true"
                pod-security.kubernetes.io/enforce: privileged
                pod-security.kubernetes.io/audit: privileged
                pod-security.kubernetes.io/warn: privileged
              name: openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1
            kind: OperatorGroup
            metadata:
              name: openshift-storage-operatorgroup
              namespace: openshift-storage
            spec:
              targetNamespaces:
              - openshift-storage
        - complianceType: musthave
          objectDefinition:
            apiVersion: operators.coreos.com/v1alpha1
            kind: Subscription
            metadata:
              name: lvms
              namespace: openshift-storage
            spec:
              installPlanApproval: Automatic
              name: lvms-operator
              source: redhat-operators
              sourceNamespace: openshift-marketplace
        remediationAction: enforce
        severity: low
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: lvms
      spec:
        object-templates:
           - complianceType: musthave
             objectDefinition:
               apiVersion: lvm.topolvm.io/v1alpha1
               kind: LVMCluster
               metadata:
                 name: my-lvmcluster
                 namespace: openshift-storage
               spec:
                 storage:
                   deviceClasses:
                   - name: vg1
                     default: true
                     deviceSelector: (1)
                       paths:
                       - /dev/disk/by-path/pci-0000:87:00.0-nvme-1
                       - /dev/disk/by-path/pci-0000:88:00.0-nvme-1
                       optionalPaths:
                       - /dev/disk/by-path/pci-0000:89:00.0-nvme-1
                       - /dev/disk/by-path/pci-0000:90:00.0-nvme-1
                     thinPoolConfig:
                       name: thin-pool-1
                       sizePercent: 90
                       overprovisionRatio: 10
                     nodeSelector:
                       nodeSelectorTerms:
                       - matchExpressions:
                           - key: app
                             operator: In
                             values:
                             - test1
        remediationAction: enforce
        severity: low
    1 Optional. To control or restrict the volume group to your preferred devices, you can manually specify the local paths of the devices in the deviceSelector section of the LVMCluster YAML. The paths section refers to devices the LVMCluster adds, which means those paths must exist. The optionalPaths section refers to devices the LVMCluster might add. You must specify at least one of paths or optionalPaths when specifying the deviceSelector section. If you specify paths, it is not mandatory to specify optionalPaths. If you specify optionalPaths, it is not mandatory to specify paths but at least one optional path must be present on the node. If you do not specify any paths, it will add all unused devices on the node.

  4. Edit the policy by running the following command:

    # oc edit -f policy-lvms-operator.yaml -ns lvms-policy-ns (1)
    1 The policy-lvms-operator.yaml is the name of the existing policy.

    This uses the new disk specified in the LVMCluster CR to provision storage.

Additional resources

Expanding PVCs

To leverage the new storage after adding additional capacity, you can expand existing persistent volume claims (PVCs) with LVM Storage.

Prerequisites

  • Dynamic provisioning is used.

  • The controlling StorageClass object has allowVolumeExpansion set to true.

Procedure

  1. Modify the .spec.resources.requests.storage field in the desired PVC resource to the new size by running the following command:

    oc patch <pvc_name> -n <application_namespace> -p '{ "spec": { "resources": { "requests": { "storage": "<desired_size>" }}}}'

  2. Watch the status.conditions field of the PVC to see if the resize has completed. OKD adds the Resizing condition to the PVC during expansion, which is removed after the expansion completes.

Additional resources

Upgrading LVM Storage on single-node OpenShift clusters

You can upgrade the Logical Volume Manager (LVM) Storage Operator to ensure compatibility with your single-node OpenShift version.

Prerequisites

  • You have upgraded your single-node OpenShift cluster.

  • You have installed a previous version of the LVM Storage Operator.

  • You have installed the OpenShift CLI (oc).

  • You have logged in as a user with cluster-admin privileges.

Procedure

  1. Update the Subscription resource for the LVM Storage Operator by running the following command:

    $ oc patch subscription lvms-operator -n openshift-storage --type merge --patch '{"spec":{"channel":"<update-channel>"}}' (1)
    1 Replace <update-channel> with the version of the LVM Storage Operator that you want to install, for example stable-4.14.

  2. View the upgrade events to check that the installation is complete by running the following command:

    $ oc get events -n openshift-storage
    Example output
    ...
8m13s       Normal    RequirementsUnknown   clusterserviceversion/lvms-operator.v4.14   requirements not yet checked
8m11s       Normal    RequirementsNotMet    clusterserviceversion/lvms-operator.v4.14   one or more requirements couldn't be found
7m50s       Normal    AllRequirementsMet    clusterserviceversion/lvms-operator.v4.14   all requirements found, attempting install
7m50s       Normal    InstallSucceeded      clusterserviceversion/lvms-operator.v4.14   waiting for install components to report healthy
7m49s       Normal    InstallWaiting        clusterserviceversion/lvms-operator.v4.14   installing: waiting for deployment lvms-operator to become ready: deployment "lvms-operator" waiting for 1 outdated replica(s) to be terminated
7m39s       Normal    InstallSucceeded      clusterserviceversion/lvms-operator.v4.14   install strategy completed with no errors
...
Verification

  • Verify the version of the LVM Storage Operator by running the following command:

    $ oc get subscription lvms-operator -n openshift-storage -o jsonpath='{.status.installedCSV}'
    Example output
    lvms-operator.v4.14

About volume snapshots

You can create snapshots of persistent volume claims (PVCs) that are provisioned by LVM Storage.

You can perform the following actions using the volume snapshots:

  • Back up your application data.

    Volume snapshots are located on the same devices as the original data. To use the volume snapshots as backups, you must move the snapshots to a secure location. You can use OpenShift API for Data Protection (OADP) backup and restore solutions. For information on OADP, see "OADP features".

  • Revert to a state at which the volume snapshot was taken.

You can also create volume snapshots of volume clones.
Additional resources

Creating volume snapshots

You can create volume snapshots based on the available capacity of the thin pool and the over-provisioning limits. To create a volume snapshot, you must create a VolumeSnapshot object.

Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

  • You ensured that the persistent volume claim (PVC) is in Bound state. This is required for a consistent snapshot.

  • You stopped all the I/O to the PVC.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Create a VolumeSnapshot object:

    Example VolumeSnapshot object
    apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: lvm-block-1-snap (1)
spec:
  source:
    persistentVolumeClaimName: lvm-block-1 (2)
  volumeSnapshotClassName: lvms-vg1 (3)
    1 Specify a name for the volume snapshot.
    2 Specify the name of the source PVC. LVM Storage creates a snapshot of this PVC.
    3 Set this field to the name of a volume snapshot class.

    To get the list of available volume snapshot classes, run the following command:

    $ oc get volumesnapshotclass

  3. Create the volume snapshot in the namespace where you created the source PVC by running the following command:

    $ oc create -f <file_name> -n <namespace>

    LVM Storage creates a read-only copy of the PVC as a volume snapshot.

Verification

  • To verify that the volume snapshot is created, run the following command:

    $ oc get volumesnapshot -n <namespace>
    Example output
    NAME               READYTOUSE   SOURCEPVC     SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS   SNAPSHOTCONTENT                                    CREATIONTIME   AGE
lvm-block-1-snap   true         lvms-test-1                           1Gi           lvms-vg1        snapcontent-af409f97-55fc-40cf-975f-71e44fa2ca91   19s            19s

    The value of the READYTOUSE field for the volume snapshot that you created must be true.

Restoring volume snapshots

To restore a volume snapshot, you must create a persistent volume claim (PVC) with the dataSource.name field set to the name of the volume snapshot.

The restored PVC is independent of the volume snapshot and the source PVC.

Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

  • You have created a volume snapshot.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Create a PersistentVolumeClaim object with the configuration to restore the volume snapshot:

    Example PersistentVolumeClaim object to restore a volume snapshot
    kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: lvm-block-1-restore
spec:
  accessModes:
  - ReadWriteOnce
  volumeMode: Block
  Resources:
    Requests:
      storage: 2Gi (1)
  storageClassName: lvms-vg1 (2)
  dataSource:
    name: lvm-block-1-snap (3)
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
    1 Specify the storage size of the PVC. The storage size of the requested PVC must be greater than or equal to the stoage size of the volume snapshot that you want to restore. If a larger PVC is required, you can also resize the PVC after restoring the volume snapshot.
    2 Set this field to the value of the storageClassName field in the source PVC of the volume snapshot that you want to restore.
    3 Set this field to the name of the volume snapshot that you want to restore.

  3. Create the PVC in the namespace where you created the the volume snapshot by running the following command:

    $ oc create -f <file_name> -n <namespace>
Verification

  • To verify that the volume snapshot is restored, create a workload using the restored PVC and then run the following command:

    $ oc get pvc -n <namespace>
    Example output
    NAME                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
lvm-block-1-restore   Bound    pvc-e90169a8-fd71-4eea-93b8-817155f60e47   1Gi        RWO            lvms-vg1       5s

Deleting volume snapshots

You can delete the volume snapshots of the persistent volume claims (PVCs).

When you delete a persistent volume claim (PVC), LVM Storage deletes only the PVC, but not the snapshots of the PVC.
Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

  • You have ensured that the volume snpashot that you want to delete is not in use.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Delete the volume snapshot by running the following command:

    $ oc delete volumesnapshot <volume_snapshot_name> -n <namespace>
Verification

  • To verify that the volume snapshot is deleted, run the following command:

    $ oc get volumesnapshot -n <namespace>

    The deleted volume snapshot must not be present in the output of this command.

About volume clones

A volume clone is a duplicate of an existing persistent volume claim (PVC). You can create a volume clone to make a point-in-time copy of the data.

Creating volume clones

To create a clone of a persistent volume claim (PVC), you must create a PersistentVolumeClaim object in the namespace where you created the source PVC.

The cloned PVC has write access.
Prerequisites

  • You ensured that the source PVC is in Bound state. This is required for a consistent clone.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Create a PersistentVolumeClaim object:

    Example PersistentVolumeClaim object to create a volume clone
    kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: lvm-pvc-clone
spec:
  accessModes:
  - ReadWriteOnce
  storageClassName: lvms-vg1 (1)
  volumeMode: Filesystem (2)
  dataSource:
    kind: PersistentVolumeClaim
    name: lvm-pvc (3)
  resources:
    requests:
      storage: 1Gi (4)
    1 Set this field to the value of the storageClassName field in the source PVC.
    2 Set this field to the volumeMode field in the source PVC.
    3 Specify the name of the source PVC.
    4 Specify the storage size for the cloned PVC. The storage size of the cloned PVC must be greater than or equal to the storage size of the source PVC.

  3. Create the PVC in the namespace where you created the source PVC by running the following command:

    $ oc create -f <file_name> -n <namespace>
Verification

  • To verify that the volume clone is created, create a workload using the cloned PVC and then run the following command:

    $ oc get pvc -n <namespace>
    Example output
    NAME                STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
lvm-block-1-clone   Bound    pvc-e90169a8-fd71-4eea-93b8-817155f60e47   1Gi        RWO            lvms-vg1       5s

Deleting volume clones

You can delete volume clones.

When you delete a persistent volume claim (PVC), LVM Storage deletes only the source persistent volume claim (PVC) but not the clones of the PVC.
Prerequisites

  • You have access to OKD as a user with cluster-admin permissions.

Procedure

  1. Log in to the OpenShift CLI (oc).

  2. Delete the cloned PVC by running the following command:

    # oc delete pvc <clone_pvc_name> -n <namespace>
Verification

  • To verify that the volume clone is deleted, run the following command:

    $ oc get pvc -n <namespace>

    The deleted volume clone must not be present in the output of this command.

Monitoring LVM Storage

To enable cluster monitoring, you must add the following label in the namespace where you have installed LVM Storage:

openshift.io/cluster-monitoring=true

For information about enabling cluster monitoring in RHACM, see Observability and Adding custom metrics.

Metrics

You can monitor LVM Storage by viewing the metrics.

The following table describes the topolvm metrics:

Table 3. topolvm metrics
Alert Description

topolvm_thinpool_data_percent

Indicates the percentage of data space used in the LVM thinpool.

topolvm_thinpool_metadata_percent

Indicates the percentage of metadata space used in the LVM thinpool.

topolvm_thinpool_size_bytes

Indicates the size of the LVM thin pool in bytes.

topolvm_volumegroup_available_bytes

Indicates the available space in the LVM volume group in bytes.

topolvm_volumegroup_size_bytes

Indicates the size of the LVM volume group in bytes.

topolvm_thinpool_overprovisioned_available

Indicates the available over-provisioned size of the LVM thin pool in bytes.

Metrics are updated every 10 minutes or when there is a change, such as a new logical volume creation, in the thin pool.

Alerts

When the thin pool and volume group reach maximum storage capacity, further operations fail. This can lead to data loss.

LVM Storage sends the following alerts when the usage of the thin pool and volume group exceeds a certain value:

Table 4. LVM Storage alerts
Alert Description

VolumeGroupUsageAtThresholdNearFull

This alert is triggered when both the volume group and thin pool usage exceeds 75% on nodes. Data deletion or volume group expansion is required.

VolumeGroupUsageAtThresholdCritical

This alert is triggered when both the volume group and thin pool usage exceeds 85% on nodes. In this case, the volume group is critically full. Data deletion or volume group expansion is required.

ThinPoolDataUsageAtThresholdNearFull

This alert is triggered when the thin pool data uusage in the volume group exceeds 75% on nodes. Data deletion or thin pool expansion is required.

ThinPoolDataUsageAtThresholdCritical

This alert is triggered when the thin pool data usage in the volume group exceeds 85% on nodes. Data deletion or thin pool expansion is required.

ThinPoolMetaDataUsageAtThresholdNearFull

This alert is triggered when the thin pool metadata usage in the volume group exceeds 75% on nodes. Data deletion or thin pool expansion is required.

ThinPoolMetaDataUsageAtThresholdCritical

This alert is triggered when the thin pool metadata usage in the volume group exceeds 85% on nodes. Data deletion or thin pool expansion is required.

Uninstalling LVM Storage using the web console

You can uninstall Logical Volume Manager (LVM) Storage using the OKD web console.

Prerequisites

  • You have access to the single-node OpenShift cluster as a user with cluster-admin permissions.

  • You have deleted the persistent volume claims (PVCs), volume snapshots, and volume clones provisioned by LVM Storage. You have also deleted the applications that are using these resources.

  • You have deleted the LVMCluster custom resource (CR).

Procedure

  1. Log in to the OKD web console.

  2. Click Operators → Installed Operators.

  3. Click LVM Storage in the openshift-storage namespace.

  4. Click the Details tab.

  5. From the Actions menu, click Uninstall Operator.

  6. Optional: When prompted, select the Delete all operand instances for this operator checkbox to delete the operand instances for LVM Storage.

  7. Click Uninstall.

Uninstalling LVM Storage installed using RHACM

To uninstall LVM Storage that you installed using RHACM, you need to delete the RHACM policy that you created for deploying and configuring the Operator.

When you delete the RHACM policy, the resources that the policy has created are not removed. You need to create additional policies to remove the resources.

As the created resources are not removed when you delete the policy, you need to perform the following steps:

  1. Remove all the Persistent volume claims (PVCs) and volume snapshots provisioned by LVM Storage.

  2. Remove the LVMCluster resources to clean up Logical Volume Manager resources created on the disks.

  3. Create an additional policy to uninstall the Operator.

Prerequisites

  • Ensure that the following are deleted before deleting the policy:

    • All the applications on the managed clusters that are using the storage provisioned by LVM Storage.

    • PVCs and persistent volumes (PVs) provisioned using LVM Storage.

    • All volume snapshots provisioned by LVM Storage.

  • Ensure you have access to the RHACM cluster using an account with a cluster-admin role.

Procedure

  1. In the OpenShift CLI (oc), delete the RHACM policy that you created for deploying and configuring LVM Storage on the hub cluster by using the following command:

    # oc delete -f policy-lvms-operator.yaml -n lvms-policy-ns (1)
    1 The policy-lvms-operator.yaml is the name of the file to which the policy was saved.

  2. To create a policy for removing the LVMCluster resource, save the following YAML to a file with a name such as lvms-remove-policy.yaml. This enables the Operator to clean up all Logical Volume Manager resources that it created on the cluster.

    apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  name: policy-lvmcluster-delete
  annotations:
    policy.open-cluster-management.io/standards: NIST SP 800-53
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
  remediationAction: enforce
  disabled: false
  policy-templates:
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: policy-lvmcluster-removal
        spec:
          remediationAction: enforce (1)
          severity: low
          object-templates:
            - complianceType: mustnothave
              objectDefinition:
                kind: LVMCluster
                apiVersion: lvm.topolvm.io/v1alpha1
                metadata:
                  name: my-lvmcluster
                  namespace: openshift-storage (2)
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-policy-lvmcluster-delete
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-policy-lvmcluster-delete
subjects:
  - apiGroup: policy.open-cluster-management.io
    kind: Policy
    name: policy-lvmcluster-delete
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-policy-lvmcluster-delete
spec:
  clusterConditions:
    - status: "True"
      type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
      - key: mykey
        operator: In
        values:
          - myvalue
    1 The policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.
    2 This namespace field must have the openshift-storage value.

  3. Set the value of the PlacementRule.spec.clusterSelector field to select the clusters from which to uninstall LVM Storage.

  4. Create the policy by running the following command:

    # oc create -f lvms-remove-policy.yaml -n lvms-policy-ns

  5. To create a policy to check if the LVMCluster CR has been removed, save the following YAML to a file with a name such as check-lvms-remove-policy.yaml:

    apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  name: policy-lvmcluster-inform
  annotations:
    policy.open-cluster-management.io/standards: NIST SP 800-53
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
spec:
  remediationAction: inform
  disabled: false
  policy-templates:
    - objectDefinition:
        apiVersion: policy.open-cluster-management.io/v1
        kind: ConfigurationPolicy
        metadata:
          name: policy-lvmcluster-removal-inform
        spec:
          remediationAction: inform (1)
          severity: low
          object-templates:
            - complianceType: mustnothave
              objectDefinition:
                kind: LVMCluster
                apiVersion: lvm.topolvm.io/v1alpha1
                metadata:
                  name: my-lvmcluster
                  namespace: openshift-storage (2)
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-policy-lvmcluster-check
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-policy-lvmcluster-check
subjects:
  - apiGroup: policy.open-cluster-management.io
    kind: Policy
    name: policy-lvmcluster-inform
---
apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-policy-lvmcluster-check
spec:
  clusterConditions:
    - status: "True"
      type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
      - key: mykey
        operator: In
        values:
          - myvalue
    1 The policy-template spec.remediationAction is overridden by the preceding parameter value for spec.remediationAction.
    2 The namespace field must have the openshift-storage value.

  6. Create the policy by running the following command:

    # oc create -f check-lvms-remove-policy.yaml -n lvms-policy-ns

  7. Check the policy status by running the following command:

    # oc get policy -n lvms-policy-ns
    Example output
    NAME                       REMEDIATION ACTION   COMPLIANCE STATE   AGE
policy-lvmcluster-delete   enforce              Compliant          15m
policy-lvmcluster-inform   inform               Compliant          15m

  8. After both the policies are compliant, save the following YAML to a file with a name such as lvms-uninstall-policy.yaml to create a policy to uninstall LVM Storage.

    apiVersion: apps.open-cluster-management.io/v1
kind: PlacementRule
metadata:
  name: placement-uninstall-lvms
spec:
  clusterConditions:
  - status: "True"
    type: ManagedClusterConditionAvailable
  clusterSelector:
    matchExpressions:
    - key: mykey
      operator: In
      values:
      - myvalue
---
apiVersion: policy.open-cluster-management.io/v1
kind: PlacementBinding
metadata:
  name: binding-uninstall-lvms
placementRef:
  apiGroup: apps.open-cluster-management.io
  kind: PlacementRule
  name: placement-uninstall-lvms
subjects:
- apiGroup: policy.open-cluster-management.io
  kind: Policy
  name: uninstall-lvms
---
apiVersion: policy.open-cluster-management.io/v1
kind: Policy
metadata:
  annotations:
    policy.open-cluster-management.io/categories: CM Configuration Management
    policy.open-cluster-management.io/controls: CM-2 Baseline Configuration
    policy.open-cluster-management.io/standards: NIST SP 800-53
  name: uninstall-lvms
spec:
  disabled: false
  policy-templates:
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: uninstall-lvms
      spec:
        object-templates:
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: v1
            kind: Namespace
            metadata:
              name: openshift-storage
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: operators.coreos.com/v1
            kind: OperatorGroup
            metadata:
              name: openshift-storage-operatorgroup
              namespace: openshift-storage
            spec:
              targetNamespaces:
              - openshift-storage
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: operators.coreos.com/v1alpha1
            kind: Subscription
            metadata:
              name: lvms-operator
              namespace: openshift-storage
        remediationAction: enforce
        severity: low
  - objectDefinition:
      apiVersion: policy.open-cluster-management.io/v1
      kind: ConfigurationPolicy
      metadata:
        name: policy-remove-lvms-crds
      spec:
        object-templates:
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: logicalvolumes.topolvm.io
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmclusters.lvm.topolvm.io
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmvolumegroupnodestatuses.lvm.topolvm.io
        - complianceType: mustnothave
          objectDefinition:
            apiVersion: apiextensions.k8s.io/v1
            kind: CustomResourceDefinition
            metadata:
              name: lvmvolumegroups.lvm.topolvm.io
        remediationAction: enforce
        severity: high

  9. Create the policy by running the following command:

    # oc create -f lvms-uninstall-policy.yaml -ns lvms-policy-ns

Downloading log files and diagnostic information using must-gather

When LVM Storage is unable to automatically resolve a problem, use the must-gather tool to collect the log files and diagnostic information so that you or the Red Hat Support can review the problem and determine a solution.

Procedure

  • Run the must-gather command from the client connected to the LVM Storage cluster:

    $ oc adm must-gather --image=registry.redhat.io/lvms4/lvms-must-gather-rhel9:v4.14 --dest-dir=<directory_name>
Additional resources