Configuring OADP with OpenShift Virtualization - OADP Application backup and restore | Backup and restore

Installing and configuring OADP with OKD Virtualization
Installing the Data Protection Application
Backing up a single VM
Restoring a single VM
Restoring a single VM from a backup of multiple VMs
Configuring the DPA with client burst and QPS settings
Overriding the imagePullPolicy setting in the DPA
- Configuring node agents and node labels
About incremental back up support

You can install the OpenShift API for Data Protection (OADP) with OKD Virtualization by installing the OADP Operator and configuring a backup location. Then, you can install the Data Protection Application.

Back up and restore virtual machines by using the OpenShift API for Data Protection.

OpenShift API for Data Protection with OKD Virtualization supports the following backup and restore storage options:

Container Storage Interface (CSI) backups
Container Storage Interface (CSI) backups with DataMover

The following storage options are excluded:

File system backup and restore
Volume snapshot backups and restores

For more information, see Backing up applications with File System Backup: Kopia or Restic.

To install the OADP Operator in a restricted network environment, you must first disable the default OperatorHub sources and mirror the Operator catalog. See Using Operator Lifecycle Manager in disconnected environments for details.

Installing and configuring OADP with OKD Virtualization

As a cluster administrator, you install OADP by installing the OADP Operator.

The latest version of the OADP Operator installs Velero 1.14.

Prerequisites

Access to the cluster as a user with the cluster-admin role.

Procedure

Install the OADP Operator according to the instructions for your storage provider.
Install the Data Protection Application (DPA) with the kubevirt and openshift OADP plugins.
Back up virtual machines by creating a Backup custom resource (CR).
Red Hat support is limited to only the following options:

CSI backups

CSI backups with DataMover.

You restore the Backup CR by creating a Restore CR.

Additional resources

Installing the Data Protection Application

You install the Data Protection Application (DPA) by creating an instance of the DataProtectionApplication API.

Prerequisites

You must install the OADP Operator.
You must configure object storage as a backup location.
If you use snapshots to back up PVs, your cloud provider must support either a native snapshot API or Container Storage Interface (CSI) snapshots.

If the backup and snapshot locations use the same credentials, you must create a Secret with the default name, cloud-credentials.

If you do not want to specify backup or snapshot locations during the installation, you can create a default Secret with an empty credentials-velero file. If there is no default Secret, the installation will fail.

Procedure

Click Operators → Installed Operators and select the OADP Operator.
Under Provided APIs, click Create instance in the DataProtectionApplication box.

Click YAML View and update the parameters of the DataProtectionApplication manifest:

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: <dpa_sample>
  namespace: openshift-adp (1)
spec:
  configuration:
    velero:
      defaultPlugins:
        - kubevirt (2)
        - gcp (3)
        - csi (4)
        - openshift (5)
      resourceTimeout: 10m (6)
    nodeAgent: (7)
      enable: true (8)
      uploaderType: kopia (9)
      podConfig:
        nodeSelector: <node_selector> (10)
  backupLocations:
    - velero:
        provider: gcp (11)
        default: true
        credential:
          key: cloud
          name: <default_secret> (12)
        objectStorage:
          bucket: <bucket_name> (13)
          prefix: <prefix> (14)

1	The default namespace for OADP is `openshift-adp`. The namespace is a variable and is configurable.
2	The `kubevirt` plugin is mandatory for OKD Virtualization.
3	Specify the plugin for the backup provider, for example, `gcp`, if it exists.
4	The `csi` plugin is mandatory for backing up PVs with CSI snapshots. The `csi` plugin uses the Velero CSI beta snapshot APIs. You do not need to configure a snapshot location.
5	The `openshift` plugin is mandatory.
6	Specify how many minutes to wait for several Velero resources before timeout occurs, such as Velero CRD availability, volumeSnapshot deletion, and backup repository availability. The default is 10m.
7	The administrative agent that routes the administrative requests to servers.
8	Set this value to `true` if you want to enable `nodeAgent` and perform File System Backup.
9	Enter `kopia` as your uploader to use the Built-in DataMover. The `nodeAgent` deploys a daemon set, which means that the `nodeAgent` pods run on each working node. You can configure File System Backup by adding `spec.defaultVolumesToFsBackup: true` to the `Backup` CR.
10	Specify the nodes on which Kopia are available. By default, Kopia runs on all nodes.
11	Specify the backup provider.
12	Specify the correct default name for the `Secret`, for example, `cloud-credentials-gcp`, if you use a default plugin for the backup provider. If specifying a custom name, then the custom name is used for the backup location. If you do not specify a `Secret` name, the default name is used.
13	Specify a bucket as the backup storage location. If the bucket is not a dedicated bucket for Velero backups, you must specify a prefix.
14	Specify a prefix for Velero backups, for example, `velero`, if the bucket is used for multiple purposes.

Click Create.

Verification

Verify the installation by viewing the OpenShift API for Data Protection (OADP) resources by running the following command:

$ oc get all -n openshift-adp

Example output

NAME                                                     READY   STATUS    RESTARTS   AGE
pod/oadp-operator-controller-manager-67d9494d47-6l8z8    2/2     Running   0          2m8s
pod/node-agent-9cq4q                                     1/1     Running   0          94s
pod/node-agent-m4lts                                     1/1     Running   0          94s
pod/node-agent-pv4kr                                     1/1     Running   0          95s
pod/velero-588db7f655-n842v                              1/1     Running   0          95s

NAME                                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/oadp-operator-controller-manager-metrics-service   ClusterIP   172.30.70.140    <none>        8443/TCP   2m8s
service/openshift-adp-velero-metrics-svc                   ClusterIP   172.30.10.0      <none>        8085/TCP   8h

NAME                        DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/node-agent    3         3         3       3            3           <none>          96s

NAME                                                READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/oadp-operator-controller-manager    1/1     1            1           2m9s
deployment.apps/velero                              1/1     1            1           96s

NAME                                                           DESIRED   CURRENT   READY   AGE
replicaset.apps/oadp-operator-controller-manager-67d9494d47    1         1         1       2m9s
replicaset.apps/velero-588db7f655                              1         1         1       96s

Verify that the DataProtectionApplication (DPA) is reconciled by running the following command:

$ oc get dpa dpa-sample -n openshift-adp -o jsonpath='{.status}'

Example output

{"conditions":[{"lastTransitionTime":"2023-10-27T01:23:57Z","message":"Reconcile complete","reason":"Complete","status":"True","type":"Reconciled"}]}

Verify the type is set to Reconciled.

Verify the backup storage location and confirm that the PHASE is Available by running the following command:

$ oc get backupstoragelocations.velero.io -n openshift-adp

Example output

NAME           PHASE       LAST VALIDATED   AGE     DEFAULT
dpa-sample-1   Available   1s               3d16h   true

If you run a backup of a Microsoft Windows virtual machine (VM) immediately after the VM reboots, the backup might fail with a PartiallyFailed error. This is because, immediately after a VM boots, the Microsoft Windows Volume Shadow Copy Service (VSS) and Guest Agent (GA) service are not ready. The VSS and GA service being unready causes the backup to fail. In such a case, retry the backup a few minutes after the VM boots.

Backing up a single VM

If you have a namespace with multiple virtual machines (VMs), and want to back up only one of them, you can use the label selector to filter the VM that needs to be included in the backup. You can filter the VM by using the app: vmname label.

Prerequisites

You have installed the OADP Operator.
You have multiple VMs running in a namespace.
You have added the kubevirt plugin in the DataProtectionApplication (DPA) custom resource (CR).
You have configured the BackupStorageLocation CR in the DataProtectionApplication CR and BackupStorageLocation is available.

Procedure

Configure the Backup CR as shown in the following example:

Example Backup CR

apiVersion: velero.io/v1
kind: Backup
metadata:
  name: vmbackupsingle
  namespace: openshift-adp
spec:
  snapshotMoveData: true
  includedNamespaces:
  - <vm_namespace> (1)
  labelSelector:
    matchLabels:
      app: <vm_app_name> (2)
  storageLocation: <backup_storage_location_name> (3)

1	Specify the name of the namespace where you have created the VMs.
2	Specify the VM name that needs to be backed up.
3	Specify the name of the `BackupStorageLocation` CR.

To create a Backup CR, run the following command:
```
$ oc apply -f <backup_cr_file_name> (1)
```
1 Specify the name of the Backup CR file.

Restoring a single VM

After you have backed up a single virtual machine (VM) by using the label selector in the Backup custom resource (CR), you can create a Restore CR and point it to the backup. This restore operation restores a single VM.

Prerequisites

You have installed the OADP Operator.
You have backed up a single VM by using the label selector.

Procedure

Configure the Restore CR as shown in the following example:

Example Restore CR

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: vmrestoresingle
  namespace: openshift-adp
spec:
  backupName: vmbackupsingle (1)
  restorePVs: true

1	Specifies the name of the backup of a single VM.

To restore the single VM, run the following command:
```
$ oc apply -f <restore_cr_file_name> (1)
```
1 Specify the name of the Restore CR file.

Restoring a single VM from a backup of multiple VMs

If you have a backup containing multiple virtual machines (VMs), and you want to restore only one VM, you can use the LabelSelectors section in the Restore CR to select the VM to restore. To ensure that the persistent volume claim (PVC) attached to the VM is correctly restored, and the restored VM is not stuck in a Provisioning status, use both the app: <vm_name> and the kubevirt.io/created-by labels. To match the kubevirt.io/created-by label, use the UID of DataVolume of the VM.

Prerequisites

You have installed the OADP Operator.
You have labeled the VMs that need to be backed up.
You have a backup of multiple VMs.

Procedure

Before you take a backup of many VMs, ensure that the VMs are labeled by running the following command:
```
$ oc label vm <vm_name> app=<vm_name> -n openshift-adp
```

Configure the label selectors in the Restore CR as shown in the following example:

Example Restore CR

apiVersion: velero.io/v1
kind: Restore
metadata:
  name: singlevmrestore
  namespace: openshift-adp
spec:
  backupName: multiplevmbackup
  restorePVs: true
  LabelSelectors:
    - matchLabels:
        kubevirt.io/created-by: <datavolume_uid> (1)
    - matchLabels:
        app: <vm_name> (2)

1	Specify the UID of `DataVolume` of the VM that you want to restore. For example, `b6…53a-ddd7-4d9d-9407-a0c…e5`.
2	Specify the name of the VM that you want to restore. For example, `test-vm`.

To restore a VM, run the following command:
```
$ oc apply -f <restore_cr_file_name> (1)
```
1 Specify the name of the Restore CR file.

Configuring the DPA with client burst and QPS settings

The burst setting determines how many requests can be sent to the velero server before the limit is applied. After the burst limit is reached, the queries per second (QPS) setting determines how many additional requests can be sent per second.

You can set the burst and QPS values of the velero server by configuring the Data Protection Application (DPA) with the burst and QPS values. You can use the dpa.configuration.velero.client-burst and dpa.configuration.velero.client-qps fields of the DPA to set the burst and QPS values.

Prerequisites

You have installed the OADP Operator.

Procedure

Configure the client-burst and the client-qps fields in the DPA as shown in the following example:

Example Data Protection Application

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: test-dpa
  namespace: openshift-adp
spec:
  backupLocations:
    - name: default
      velero:
        config:
          insecureSkipTLSVerify: "true"
          profile: "default"
          region: <bucket_region>
          s3ForcePathStyle: "true"
          s3Url: <bucket_url>
        credential:
          key: cloud
          name: cloud-credentials
        default: true
        objectStorage:
          bucket: <bucket_name>
          prefix: velero
        provider: aws
  configuration:
    nodeAgent:
      enable: true
      uploaderType: restic
    velero:
      client-burst: 500 (1)
      client-qps: 300 (2)
      defaultPlugins:
        - openshift
        - aws
        - kubevirt

1	Specify the `client-burst` value. In this example, the `client-burst` field is set to 500.
2	Specify the `client-qps` value. In this example, the `client-qps` field is set to 300.

Overriding the imagePullPolicy setting in the DPA

In OADP 1.4.0 or earlier, the Operator sets the imagePullPolicy field of the Velero and node agent pods to Always for all images.

In OADP 1.4.1 or later, the Operator first checks if each image has the sha256 or sha512 digest and sets the imagePullPolicy field accordingly:

If the image has the digest, the Operator sets imagePullPolicy to IfNotPresent.
If the image does not have the digest, the Operator sets imagePullPolicy to Always.

You can also override the imagePullPolicy field by using the spec.imagePullPolicy field in the Data Protection Application (DPA).

Prerequisites

You have installed the OADP Operator.

Procedure

Configure the spec.imagePullPolicy field in the DPA as shown in the following example:

Example Data Protection Application

apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
  name: test-dpa
  namespace: openshift-adp
spec:
  backupLocations:
    - name: default
      velero:
        config:
          insecureSkipTLSVerify: "true"
          profile: "default"
          region: <bucket_region>
          s3ForcePathStyle: "true"
          s3Url: <bucket_url>
        credential:
          key: cloud
          name: cloud-credentials
        default: true
        objectStorage:
          bucket: <bucket_name>
          prefix: velero
        provider: aws
  configuration:
    nodeAgent:
      enable: true
      uploaderType: kopia
    velero:
      defaultPlugins:
        - openshift
        - aws
        - kubevirt
        - csi
  imagePullPolicy: Never (1)

1	Specify the value for `imagePullPolicy`. In this example, the `imagePullPolicy` field is set to `Never`.

Configuring node agents and node labels

The Data Protection Application (DPA) uses the nodeSelector field to select which nodes can run the node agent. The nodeSelector field is the recommended form of node selection constraint.

Procedure

Run the node agent on any node that you choose by adding a custom label:
```
$ oc label node/<node_name> node-role.kubernetes.io/nodeAgent=""
```
Any label specified must match the labels on each node.

Use the same custom label in the DPA.spec.configuration.nodeAgent.podConfig.nodeSelector field, which you used for labeling nodes:

configuration:
  nodeAgent:
    enable: true
    podConfig:
      nodeSelector:
        node-role.kubernetes.io/nodeAgent: ""

The following example is an anti-pattern of nodeSelector and does not work unless both labels, node-role.kubernetes.io/infra: "" and node-role.kubernetes.io/worker: "", are on the node:

    configuration:
      nodeAgent:
        enable: true
        podConfig:
          nodeSelector:
            node-role.kubernetes.io/infra: ""
            node-role.kubernetes.io/worker: ""

About incremental back up support

OADP supports incremental backups of block and Filesystem persistent volumes for both containerized, and OKD Virtualization workloads. The following table summarizes the support for File System Backup (FSB), Container Storage Interface (CSI), and CSI Data Mover:

Table 1. OADP backup support matrix for containerized workloads
Volume mode	FSB - Restic	FSB - Kopia	CSI	CSI Data Mover
Filesystem	S ^[1], I ^[2]	S ^[1], I ^[2]	S ^[1]	S ^[1], I ^[2]
Block	N ^[3]	N ^[3]	S ^[1]	S ^[1], I ^[2]

Table 2. OADP backup support matrix for OKD Virtualization workloads
Volume mode	FSB - Restic	FSB - Kopia	CSI	CSI Data Mover
Filesystem	N ^[3]	N ^[3]	S ^[1]	S ^[1], I ^[2]
Block	N ^[3]	N ^[3]	S ^[1]	S ^[1], I ^[2]

Backup supported
Incremental backup supported
Not supported

The CSI Data Mover backups use Kopia regardless of uploaderType.

Red Hat only supports the combination of OADP versions 1.3.0 and later, and OKD Virtualization versions 4.14 and later.

OADP versions before 1.3.0 are not supported for back up and restore of OKD Virtualization.

Configuring the OpenShift API for Data Protection with OKD Virtualization

Installing and configuring OADP with OKD Virtualization

Installing the Data Protection Application

Backing up a single VM

Restoring a single VM

Restoring a single VM from a backup of multiple VMs

Configuring the DPA with client burst and QPS settings

Overriding the imagePullPolicy setting in the DPA

Configuring node agents and node labels

About incremental back up support