×

You restore application backups by creating a Restore custom resource (CR). See Creating a Restore CR.

You can create restore hooks to run commands in a container in a pod by editing the Restore CR. See Creating restore hooks.

Previewing resources before running backup and restore

OADP backs up application resources based on the type, namespace, or label. This means that you can view the resources after the backup is complete. Similarly, you can view the restored objects based on the namespace, persistent volume (PV), or label after a restore operation is complete. To preview the resources in advance, you can do a dry run of the backup and restore operations.

Prerequisites
  • You have installed the OADP Operator.

Procedure
  1. To preview the resources included in the backup before running the actual backup, run the following command:

    $ velero backup create <backup-name> --snapshot-volumes false (1)
    1 Specify the value of --snapshot-volumes parameter as false.
  2. To know more details about the backup resources, run the following command:

    $ velero describe backup <backup_name> --details (1)
    1 Specify the name of the backup.
  3. To preview the resources included in the restore before running the actual restore, run the following command:

    $ velero restore create --from-backup <backup-name> (1)
    1 Specify the name of the backup created to review the backup resources.

    The velero restore create command creates restore resources in the cluster. You must delete the resources created as part of the restore, after you review the resources.

  4. To know more details about the restore resources, run the following command:

    $ velero describe restore <restore_name> --details (1)
    1 Specify the name of the restore.

Creating a Restore CR

You restore a Backup custom resource (CR) by creating a Restore CR.

Prerequisites
  • You must install the OpenShift API for Data Protection (OADP) Operator.

  • The DataProtectionApplication CR must be in a Ready state.

  • You must have a Velero Backup CR.

  • The persistent volume (PV) capacity must match the requested size at backup time. Adjust the requested size if needed.

Procedure
  1. Create a Restore CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Restore
    metadata:
      name: <restore>
      namespace: openshift-adp
    spec:
      backupName: <backup> (1)
      includedResources: [] (2)
      excludedResources:
      - nodes
      - events
      - events.events.k8s.io
      - backups.velero.io
      - restores.velero.io
      - resticrepositories.velero.io
      restorePVs: true (3)
    1 Name of the Backup CR.
    2 Optional: Specify an array of resources to include in the restore process. Resources might be shortcuts (for example, po for pods) or fully-qualified. If unspecified, all resources are included.
    3 Optional: The restorePVs parameter can be set to false to turn off restore of PersistentVolumes from VolumeSnapshot of Container Storage Interface (CSI) snapshots or from native snapshots when VolumeSnapshotLocation is configured.
  2. Verify that the status of the Restore CR is Completed by entering the following command:

    $ oc get restores.velero.io -n openshift-adp <restore> -o jsonpath='{.status.phase}'
  3. Verify that the backup resources have been restored by entering the following command:

    $ oc get all -n <namespace> (1)
    1 Namespace that you backed up.
  4. If you restore DeploymentConfig with volumes or if you use post-restore hooks, run the dc-post-restore.sh cleanup script by entering the following command:

    $ bash dc-restic-post-restore.sh -> dc-post-restore.sh

    During the restore process, the OADP Velero plug-ins scale down the DeploymentConfig objects and restore the pods as standalone pods. This is done to prevent the cluster from deleting the restored DeploymentConfig pods immediately on restore and to allow the restore and post-restore hooks to complete their actions on the restored pods. The cleanup script shown below removes these disconnected pods and scales any DeploymentConfig objects back up to the appropriate number of replicas.

    dc-restic-post-restore.sh → dc-post-restore.sh cleanup script
    #!/bin/bash
    set -e
    
    # if sha256sum exists, use it to check the integrity of the file
    if command -v sha256sum >/dev/null 2>&1; then
      CHECKSUM_CMD="sha256sum"
    else
      CHECKSUM_CMD="shasum -a 256"
    fi
    
    label_name () {
        if [ "${#1}" -le "63" ]; then
    	echo $1
    	return
        fi
        sha=$(echo -n $1|$CHECKSUM_CMD)
        echo "${1:0:57}${sha:0:6}"
    }
    
    if [[ $# -ne 1 ]]; then
        echo "usage: ${BASH_SOURCE} restore-name"
        exit 1
    fi
    
    echo "restore: $1"
    
    label=$(label_name $1)
    echo "label:   $label"
    
    echo Deleting disconnected restore pods
    oc delete pods --all-namespaces -l oadp.openshift.io/disconnected-from-dc=$label
    
    for dc in $(oc get dc --all-namespaces -l oadp.openshift.io/replicas-modified=$label -o jsonpath='{range .items[*]}{.metadata.namespace}{","}{.metadata.name}{","}{.metadata.annotations.oadp\.openshift\.io/original-replicas}{","}{.metadata.annotations.oadp\.openshift\.io/original-paused}{"\n"}')
    do
        IFS=',' read -ra dc_arr <<< "$dc"
        if [ ${#dc_arr[0]} -gt 0 ]; then
    	echo Found deployment ${dc_arr[0]}/${dc_arr[1]}, setting replicas: ${dc_arr[2]}, paused: ${dc_arr[3]}
    	cat <<EOF | oc patch dc  -n ${dc_arr[0]} ${dc_arr[1]} --patch-file /dev/stdin
    spec:
      replicas: ${dc_arr[2]}
      paused: ${dc_arr[3]}
    EOF
        fi
    done

Creating restore hooks

You create restore hooks to run commands in a container in a pod by editing the Restore custom resource (CR).

You can create two types of restore hooks:

  • An init hook adds an init container to a pod to perform setup tasks before the application container starts.

    If you restore a Restic backup, the restic-wait init container is added before the restore hook init container.

  • An exec hook runs commands or scripts in a container of a restored pod.

Procedure
  • Add a hook to the spec.hooks block of the Restore CR, as in the following example:

    apiVersion: velero.io/v1
    kind: Restore
    metadata:
      name: <restore>
      namespace: openshift-adp
    spec:
      hooks:
        resources:
          - name: <hook_name>
            includedNamespaces:
            - <namespace> (1)
            excludedNamespaces:
            - <namespace>
            includedResources:
            - pods (2)
            excludedResources: []
            labelSelector: (3)
              matchLabels:
                app: velero
                component: server
            postHooks:
            - init:
                initContainers:
                - name: restore-hook-init
                  image: alpine:latest
                  volumeMounts:
                  - mountPath: /restores/pvc1-vm
                    name: pvc1-vm
                  command:
                  - /bin/ash
                  - -c
                timeout: (4)
            - exec:
                container: <container> (5)
                command:
                - /bin/bash (6)
                - -c
                - "psql < /backup/backup.sql"
                waitTimeout: 5m (7)
                execTimeout: 1m (8)
                onError: Continue (9)
    1 Optional: Array of namespaces to which the hook applies. If this value is not specified, the hook applies to all namespaces.
    2 Currently, pods are the only supported resource that hooks can apply to.
    3 Optional: This hook only applies to objects matching the label selector.
    4 Optional: Timeout specifies the maximum length of time Velero waits for initContainers to complete.
    5 Optional: If the container is not specified, the command runs in the first container in the pod.
    6 This is the entrypoint for the init container being added.
    7 Optional: How long to wait for a container to become ready. This should be long enough for the container to start and for any preceding hooks in the same container to complete. If not set, the restore process waits indefinitely.
    8 Optional: How long to wait for the commands to run. The default is 30s.
    9 Allowed values for error handling are Fail and Continue:
    • Continue: Only command failures are logged.

    • Fail: No more restore hooks run in any container in any pod. The status of the Restore CR will be PartiallyFailed.

During a File System Backup (FSB) restore operation, a Deployment resource referencing an ImageStream is not restored properly. The restored pod that runs the FSB, and the postHook is terminated prematurely.

This happens because, during the restore operation, OpenShift controller updates the spec.template.spec.containers[0].image field in the Deployment resource with an updated ImageStreamTag hash. The update triggers the rollout of a new pod, terminating the pod on which velero runs the FSB and the post restore hook. For more information about image stream trigger, see "Triggering updates on image stream changes".

The workaround for this behavior is a two-step restore process:

  1. First, perform a restore excluding the Deployment resources, for example:

    $ velero restore create <RESTORE_NAME> \
      --from-backup <BACKUP_NAME> \
      --exclude-resources=deployment.apps
  2. After the first restore is successful, perform a second restore by including these resources, for example:

    $ velero restore create <RESTORE_NAME> \
      --from-backup <BACKUP_NAME> \
      --include-resources=deployment.apps