Soon, all Red Hat OpenShift documentation will only be available on docs.redhat.com, the official location of all Red Hat product documentation. Get familiar with the updated experience today!
  1. Documentation
×

Backing up and restoring CSI snapshots data movement

Backing up persistent volumes with CSI snapshots

You can use the OADP Data Mover to back up Container Storage Interface (CSI) volume snapshots to a remote object store.

Prerequisites

  • You have access to the cluster with the cluster-admin role.

  • You have installed the OADP Operator.

  • You have included the CSI plugin and enabled the node agent in the DataProtectionApplication custom resource (CR).

  • You have an application with persistent volumes running in a separate namespace.

  • You have added the metadata.labels.velero.io/csi-volumesnapshot-class: "true" key-value pair to the VolumeSnapshotClass CR.

Procedure

  1. Create a YAML file for the Backup object, as in the following example:

    Example Backup CR
    kind: Backup
apiVersion: velero.io/v1
metadata:
  name: backup
  namespace: openshift-adp
spec:
  csiSnapshotTimeout: 10m0s
  defaultVolumesToFsBackup: (1)
  includedNamespaces:
  - mysql-persistent
  itemOperationTimeout: 4h0m0s
  snapshotMoveData: true (2)
  storageLocation: default
  ttl: 720h0m0s (3)
  volumeSnapshotLocations:
  - dpa-sample-1
# ...
    1 Set to true if you use Data Mover only for volumes that opt out of fs-backup. Set to false if you use Data Mover by default for volumes.
    2 Set to true to enable movement of CSI snapshots to remote object storage.
    3 The ttl field defines the retention time of the created backup and the backed up data. For example, if you are using Restic as the backup tool, the backed up data items and data contents of the persistent volumes (PVs) are stored until the backup expires. But storing this data consumes more space in the target backup locations. An additional storage is consumed with frequent backups, which are created even before other unexpired completed backups might have timed out.

    If you format the volume by using XFS filesystem and the volume is at 100% capacity, the backup fails with a no space left on device error. For example:

    Error: relabel failed /var/lib/kubelet/pods/3ac..34/volumes/ \
kubernetes.io~csi/pvc-684..12c/mount: lsetxattr /var/lib/kubelet/ \
pods/3ac..34/volumes/kubernetes.io~csi/pvc-68..2c/mount/data-xfs-103: \
no space left on device

    In this scenario, consider resizing the volume or using a different filesystem type, for example, ext4, so that the backup completes successfully.

  2. Apply the manifest:

    $ oc create -f backup.yaml

    A DataUpload CR is created after the snapshot creation is complete.

Verification

  • Verify that the snapshot data is successfully transferred to the remote object store by monitoring the status.phase field of the DataUpload CR. Possible values are In Progress, Completed, Failed, or Canceled. The object store is configured in the backupLocations stanza of the DataProtectionApplication CR.

    • Run the following command to get a list of all DataUpload objects:

      $ oc get datauploads -A
      Example output
      NAMESPACE       NAME                  STATUS      STARTED   BYTES DONE   TOTAL BYTES   STORAGE LOCATION   AGE     NODE
openshift-adp   backup-test-1-sw76b   Completed   9m47s     108104082    108104082     dpa-sample-1       9m47s   ip-10-0-150-57.us-west-2.compute.internal
openshift-adp   mongo-block-7dtpf     Completed   14m       1073741824   1073741824    dpa-sample-1       14m     ip-10-0-150-57.us-west-2.compute.internal

    • Check the value of the status.phase field of the specific DataUpload object by running the following command:

      $ oc get datauploads <dataupload_name> -o yaml
      Example output
      apiVersion: velero.io/v2alpha1
kind: DataUpload
metadata:
  name: backup-test-1-sw76b
  namespace: openshift-adp
spec:
  backupStorageLocation: dpa-sample-1
  csiSnapshot:
    snapshotClass: ""
    storageClass: gp3-csi
    volumeSnapshot: velero-mysql-fq8sl
  operationTimeout: 10m0s
  snapshotType: CSI
  sourceNamespace: mysql-persistent
  sourcePVC: mysql
status:
  completionTimestamp: "2023-11-02T16:57:02Z"
  node: ip-10-0-150-57.us-west-2.compute.internal
  path: /host_pods/15116bac-cc01-4d9b-8ee7-609c3bef6bde/volumes/kubernetes.io~csi/pvc-eead8167-556b-461a-b3ec-441749e291c4/mount
  phase: Completed (1)
  progress:
    bytesDone: 108104082
    totalBytes: 108104082
  snapshotID: 8da1c5febf25225f4577ada2aeb9f899
  startTimestamp: "2023-11-02T16:56:22Z"
      1 Indicates that snapshot data is successfully transferred to the remote object store.

Restoring CSI volume snapshots

You can restore a volume snapshot by creating a Restore CR.

You cannot restore Volsync backups from OADP 1.2 with the OAPD 1.3 built-in Data Mover. It is recommended to do a file system backup of all of your workloads with Restic prior to upgrading to OADP 1.3.
Prerequisites

  • You have access to the cluster with the cluster-admin role.

  • You have an OADP Backup CR from which to restore the data.

Procedure

  1. Create a YAML file for the Restore CR, as in the following example:

    Example Restore CR
    apiVersion: velero.io/v1
kind: Restore
metadata:
  name: restore
  namespace: openshift-adp
spec:
  backupName: <backup>
# ...

  2. Apply the manifest:

    $ oc create -f restore.yaml

    A DataDownload CR is created when the restore starts.

Verification

  • You can monitor the status of the restore process by checking the status.phase field of the DataDownload CR. Possible values are In Progress, Completed, Failed, or Canceled.

    • To get a list of all DataDownload objects, run the following command:

      $ oc get datadownloads -A
      Example output
      NAMESPACE       NAME                   STATUS      STARTED   BYTES DONE   TOTAL BYTES   STORAGE LOCATION   AGE     NODE
openshift-adp   restore-test-1-sk7lg   Completed   7m11s     108104082    108104082     dpa-sample-1       7m11s   ip-10-0-150-57.us-west-2.compute.internal

    • Enter the following command to check the value of the status.phase field of the specific DataDownload object:

      $ oc get datadownloads <datadownload_name> -o yaml
      Example output
      apiVersion: velero.io/v2alpha1
kind: DataDownload
metadata:
  name: restore-test-1-sk7lg
  namespace: openshift-adp
spec:
  backupStorageLocation: dpa-sample-1
  operationTimeout: 10m0s
  snapshotID: 8da1c5febf25225f4577ada2aeb9f899
  sourceNamespace: mysql-persistent
  targetVolume:
    namespace: mysql-persistent
    pv: ""
    pvc: mysql
status:
  completionTimestamp: "2023-11-02T17:01:24Z"
  node: ip-10-0-150-57.us-west-2.compute.internal
  phase: Completed (1)
  progress:
    bytesDone: 108104082
    totalBytes: 108104082
  startTimestamp: "2023-11-02T17:00:52Z"
      1 Indicates that the CSI snapshot data is successfully restored.

Deletion policy for OADP 1.3

The deletion policy determines rules for removing data from a system, specifying when and how deletion occurs based on factors such as retention periods, data sensitivity, and compliance requirements. It manages data removal effectively while meeting regulations and preserving valuable information.

Deletion policy guidelines for OADP 1.3

Review the following deletion policy guidelines for the OADP 1.3:

  • In OADP 1.3.x, when using any type of backup and restore methods, you can set the deletionPolicy field to Retain or Delete in the VolumeSnapshotClass custom resource (CR).