OADP 1.5 release notes

Before this update, OADP would only process the first custom CA certificate found among all backup storage locations (BSLs) and apply it globally. This behavior prevented multiple BSLs with different CA certificates from working correctly. Additionally, system-trusted certificates were not included, causing failures when connecting to standard services.

The CloudStorage API feature, available as a Technology Preview before this update, is fully supported from OADP 1.5.1. The CloudStorage API automates the creation of a bucket for object storage.

The DataProtectionTest (DPT) is a custom resource (CR) that provides a framework to validate your OADP configuration.

With this update, users can control the maximum number of node agent operations that can run simultaneously on each node within their cluster. It also enables better resource management, optimizing backup and restore workflows for improved performance and a more streamlined experience.

Before this update, the DataProtectionApplicationSpec used deprecated PodAnnotations, which led to an annotation limit overflow. This caused potential misconfigurations in deployments. In this release, we have added PodConfig for annotations in pods deployed by the Operator, ensuring consistent annotations and improved manageability for end users. As a result, deployments should now be more reliable and easier to manage.

Before this update, the manager container of the openshift-adp-controller-manager-* pod was configured to run with a writable root file system. As a consequence, this could allow for tampering with the container’s file system or the writing of foreign executables. With this release, the container’s security context has been updated to set the root file system to read-only while ensuring necessary functions that require write access, such as the Kopia cache, continue to operate correctly. As a result, the container is hardened against potential threats.

Before this update, when a user attempted to create a second non-admin DataProtectionApplication (DPA) on a cluster where one already existed, the new DPA failed to reconcile. With this release, the restriction on Non-Admin Controller installation to one per cluster has been removed. As a result, users can install multiple Non-Admin Controllers across the cluster without encountering errors.

Before this update, using a self-signed certificate for backup images with a storage provider such as Minio resulted in an x509: certificate signed by unknown authority error during the backup process. With this release, certificate validation has been updated to support self-signed certificates in OADP, ensuring successful backups.

Before this update, the velero describe output command omitted the defaultVolumesToFsBackup flag. This affected the visibility of backup configuration details for users. With this release, the velero describe output includes the defaultVolumesToFsBackup flag information, improving the visibility of backup settings.

Before this update, DataProtectionTest (DPT) failed to run when s3Url was secured due to an unverified certificate because the DPT CR lacked the ability to skip or add the caCert in the spec field. As a consequence, data upload failure occurred due to an unverified certificate. With this release, DPT CR has been updated to accept and skip CA cert in spec field, resolving SSL verification errors. As a result, DPT no longer fails when using secured s3Url.

Before this update, adding a second backupLocation with the same name in DataProtectionApplication (DPA) caused OADP to enter an invalid state, leading to Backup and Restore failures due to Velero’s inability to read Secret credentials. As a consequence, Backup and Restore operations failed. With this release, the duplicate backupLocation names in DPA are no longer allowed, preventing Backup and Restore failures. As a result, duplicate backupLocation names are rejected, ensuring seamless data protection.

When you start a restore operation for a backup that was created on an OpenStack platform using the Cinder Container Storage Interface (CSI) driver, the initial backup only succeeds after the source application is manually scaled down. The restore job fails, preventing you from successfully recovering your application’s data and state from the backup. No known workaround exists.

Due to an issue in Velero 1.14 and later, the OADP node-agent only processes the first nodeSelector element within the loadAffinity array. As a consequence, if you define multiple nodeSelector objects, all objects except the first are ignored, potentially causing datamover pods to be scheduled on unexpected nodes during a backup.

The CA certificate is not stored as a file on the running Velero container. As a consequence, the user experience degraded due to missing caCert in Velero container, requiring manual setup and downloads. To work around this problem, manually add cert to the Velero deployment. For instructions, see Using cacert with velero command aliased via velero deployment.

When a Data Protection Application (DPA) is created with the nodeSelector field set in the nodeAgent parameter, Data Mover restore partially fails instead of completing the restore operation. No known workaround exists.

An unverified certificate in the S3 connection during backups with caCert in DataProtectionApplication (DPA) causes the ocp-django application’s backup to partially fail and result in data loss. No known workaround exists.

When the ephemeral-storage parameter is configured and running file system restore, the cache is not automatically deleted from the worker node. As a consequence, the /var partition overflows during backup restore, causing increased storage usage and potential resource exhaustion. To work around this problem, restart the node agent pod, which clears the cache. As a result, cache is deleted.

When performing a volumeSnapshotLocation (VSL) backup on Google Cloud Workload Identity, the Velero Google Cloud plugin creates an invalid API request if the Google Cloud project is also specified in the snapshotLocations configuration of DataProtectionApplication (DPA). As a consequence, the Google Cloud API returns a RESOURCE_PROJECT_INVALID error, and the backup job finishes with a PartiallyFailed status. No known workaround exists.

The volumeSnapshotLocation (VSL) backup fails because of missing the AZURE_RESOURCE_GROUP parameter in the credentials file, even if AZURE_RESOURCE_GROUP is already mentioned in the DataProtectionApplication (DPA) config for VSL. No known workaround exists.

When backing up applications that include image stream resources on an Azure cluster using STS, the OADP plugin incorrectly attempts to locate a secret-based credential for the container registry. As a consequence, the required secret is not found in the STS environment, causing the ImageStream custom backup action to fail. This results in the overall backup status marked as PartiallyFailed. No known workaround exists.

When a user creates a bucket and uses the backupLocations.bucket.cloudStorageRef configuration, bucket credentials are not present in the DataProtectionApplication (DPA) custom resource (CR). As a result, the DPA reconciliation fails even if bucket credentials are present in the CloudStorage CR. To work around this problem, add the same credentials to the backupLocations section of the DPA CR.

With OADP 1.5.0, the configuration.restic specification field has been deprecated. Use the nodeAgent section with the uploaderType field for selecting kopia or restic as a uploaderType. Note that Restic is deprecated in OADP 1.5.0.

OADP 1.5.0 introduces a new feature named OADP Self-Service, enabling namespace admin users to back up and restore applications on the OKD. In the earlier versions of OADP, you needed the cluster-admin role to perform OADP operations such as backing up and restoring an application, creating a backup storage location, and so on.

You can collect logs, and information about OpenShift API for Data Protection (OADP) custom resources by using the must-gather tool. The must-gather data must be attached to all customer cases. This tool generates a Markdown output file with the collected information, which is located in the must-gather logs clusters directory.

The nodeAgent field in Data Protection Application (DPA) now includes the following parameters:

Velero no longer uses the node-agent-config config map for configuring the nodeAgent daemon set. With this update, you must use the new spec.configuration.nodeAgent parameter in a Data Protection Application (DPA) for configuring the nodeAgent daemon set.

With Velero 1.15 and later, you can now configure the total size of a cache per repository. This prevents pods from being removed due to running out of ephemeral storage. See the following new parameters added to the NodeAgentConfig field in DPA:

With this update, the following changes are added:

By default, only one thread processes an item block. Velero 1.16 supports a parallel item backup, where multiple items within a backup can be processed in parallel.

With the of release OADP 1.5.0, the logs are now available in the JSON format. It helps to have pre-parsed data in their Elastic logs management system.

With this release, the oc get dpa command now displays RECONCILED status instead of displaying only NAME and AGE to improve user experience. For example:

$ oc get dpa -n openshift-adp
NAME            RECONCILED   AGE
velero-sample   True         2m51s

With this release, the terminationMessagePolicy field can now set the FallbackToLogsOnError value for the OpenShift API for Data Protection (OADP) Operator containers such as operator-manager, velero, node-agent, and non-admin-controller.

Previously, the namespace admin could not execute an application after the restore operation.

Previously, status restoration was only configured at the resource type using the restoreStatus field in the Restore custom resource (CR).

metadata:
  annotations:
    velero.io/restore-status: "true"

Previously, on performing the backup of an application with the excludedClusterScopedResources field set to storageclasses, Namespace parameter, the backup was successful but the restore partially failed. With this update, the restore is successful.

Previously, a backup was marked as failed with the following error message:

failureReason: found a backup with status "InProgress" during the server starting,
mark it as "Failed"

Previously, when the spec.configuration.velero.disableFsBackup field from a Data Protection Application (DPA) was set to true, the backup partially failed with an error, which was not informative.

Previously, AWS credentials using STS authentication were not properly validated.

Previously, the new configurations for repository maintenance job pod affinity was missing from a DPA specification.

Previously, when a schedule CR was created with a wrong spec.schedule value and the same was later patched with a correct value, the ValidationErrors field still existed. Consequently, the ValidationErrors field was displaying incorrect information even though the spec was correct.

Previously, when a restore operation was triggered with the includedNamespace field in a restore specification, restore operation was completed successfully but no volumeSnapshotContents custom resources (CR) were created and the PVCs were in a Pending status.

Previously, the container was configured with the readOnlyRootFilesystem: true setting for security, but the code attempted to create temporary files in the /tmp directory using the os.CreateTemp() function. Consequently, while using the AWS STS authentication with the Cloud Credential Operator (CCO) flow, OADP failed to create temporary files that were required for AWS credential handling with the following error:

ERROR unable to determine if bucket exists. {"error": "open /tmp/aws-shared-credentials1211864681: read-only file system"}

Even after deleting a backup, Kopia does not delete the volume artifacts from the ${bucket_name}/kopia/${namespace} on the S3 location after the backup expired. Information related to the expired and removed data files remains in the metadata.

With OpenShift API for Data Protection (OADP) 1.5.0, the configuration.restic specification field has been deprecated. Use the nodeAgent section with the uploaderType field for selecting kopia or restic as a uploaderType. Note that Restic is deprecated in OpenShift API for Data Protection (OADP) 1.5.0.

OADP can support and facilitate application migrations within HyperShift hosted OpenShift clusters as a Technology Preview. It ensures a seamless backup and restore operation for applications in hosted clusters.