1. Documentation
×

Overriding Kopia hashing, encryption, and splitter algorithms

You can override the default values of Kopia hashing, encryption, and splitter algorithms by using specific environment variables in the Data Protection Application (DPA).

Configuring the DPA to override Kopia hashing, encryption, and splitter algorithms

You can use an OpenShift API for Data Protection (OADP) option to override the default Kopia algorithms for hashing, encryption, and splitter to improve Kopia performance or to compare performance metrics. You can set the following environment variables in the spec.configuration.velero.podConfig.env section of the DPA:

  • KOPIA_HASHING_ALGORITHM

  • KOPIA_ENCRYPTION_ALGORITHM

  • KOPIA_SPLITTER_ALGORITHM

Prerequisites

  • You have installed the OADP Operator.

  • You have created the secret by using the credentials provided by the cloud provider.

The configuration of the Kopia algorithms for splitting, hashing, and encryption in the Data Protection Application (DPA) apply only during the initial Kopia repository creation, and cannot be changed later.

To use different Kopia algorithms, ensure that the object storage does not contain any previous Kopia repositories of backups. Configure a new object storage in the Backup Storage Location (BSL) or specify a unique prefix for the object storage in the BSL configuration.
Procedure

  • Configure the DPA with the environment variables for hashing, encryption, and splitter as shown in the following example.

    Example DPA
    apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
#...
configuration:
  nodeAgent:
    enable: true (1)
    uploaderType: kopia (2)
  velero:
    defaultPlugins:
    - openshift
    - aws
    - csi (3)
    defaultSnapshotMoveData: true
    podConfig:
      env:
        - name: KOPIA_HASHING_ALGORITHM
          value: <hashing_algorithm_name> (4)
        - name: KOPIA_ENCRYPTION_ALGORITHM
          value: <encryption_algorithm_name> (5)
        - name: KOPIA_SPLITTER_ALGORITHM
          value: <splitter_algorithm_name> (6)
    1 Enable the nodeAgent.
    2 Specify the uploaderType as kopia.
    3 Include the csi plugin.
    4 Specify a hashing algorithm. For example, BLAKE3-256.
    5 Specify an encryption algorithm. For example, CHACHA20-POLY1305-HMAC-SHA256.
    6 Specify a splitter algorithm. For example, DYNAMIC-8M-RABINKARP.

Use case for overriding Kopia hashing, encryption, and splitter algorithms

The use case example demonstrates taking a backup of an application by using Kopia environment variables for hashing, encryption, and splitter. You store the backup in an AWS S3 bucket. You then verify the environment variables by connecting to the Kopia repository.

Prerequisites

  • You have installed the OADP Operator.

  • You have an AWS S3 bucket configured as the backup storage location.

  • You have created the secret by using the credentials provided by the cloud provider.

  • You have installed the Kopia client.

  • You have an application with persistent volumes running in a separate namespace.

Procedure

  1. Configure the Data Protection Application (DPA) as shown in the following example:

    apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
name: <dpa_name> (1)
namespace: openshift-adp
spec:
backupLocations:
- name: aws
  velero:
    config:
      profile: default
      region: <region_name> (2)
    credential:
      key: cloud
      name: cloud-credentials (3)
    default: true
    objectStorage:
      bucket: <bucket_name> (4)
      prefix: velero
    provider: aws
configuration:
  nodeAgent:
    enable: true
    uploaderType: kopia
  velero:
    defaultPlugins:
    - openshift
    - aws
    - csi (5)
    defaultSnapshotMoveData: true
    podConfig:
      env:
        - name: KOPIA_HASHING_ALGORITHM
          value: BLAKE3-256 (6)
        - name: KOPIA_ENCRYPTION_ALGORITHM
          value: CHACHA20-POLY1305-HMAC-SHA256 (7)
        - name: KOPIA_SPLITTER_ALGORITHM
          value: DYNAMIC-8M-RABINKARP (8)
    1 Specify a name for the DPA.
    2 Specify the region for the backup storage location.
    3 Specify the name of the default Secret object.
    4 Specify the AWS S3 bucket name.
    5 Include the csi plugin.
    6 Specify the hashing algorithm as BLAKE3-256.
    7 Specify the encryption algorithm as CHACHA20-POLY1305-HMAC-SHA256.
    8 Specify the splitter algorithm as DYNAMIC-8M-RABINKARP.

  2. Create the DPA by running the following command:

    $ oc create -f <dpa_file_name> (1)
    1 Specify the file name of the DPA you configured.

  3. Verify that the DPA has reconciled by running the following command:

    $ oc get dpa -o yaml

  4. Create a backup CR as shown in the following example:

    Example backup CR
    apiVersion: velero.io/v1
kind: Backup
metadata:
  name: test-backup
  namespace: openshift-adp
spec:
  includedNamespaces:
  - <application_namespace> (1)
  defaultVolumesToFsBackup: true
    1 Specify the namespace for the application installed in the cluster.

  5. Create a backup by running the following command:

    $ oc apply -f <backup_file_name> (1)
    1 Specify the name of the backup CR file.

  6. Verify that the backup completed by running the following command:

    $ oc get backups.velero.io <backup_name> -o yaml (1)
    1 Specify the name of the backup.
Verification

  1. Connect to the Kopia repository by running the following command:

    $ kopia repository connect s3 \
  --bucket=<bucket_name> \ (1)
  --prefix=velero/kopia/<application_namespace> \ (2)
  --password=static-passw0rd \ (3)
  --access-key="<aws_s3_access_key>" \ (4)
  --secret-access-key="<aws_s3_secret_access_key>" \ (5)
    1 Specify the AWS S3 bucket name.
    2 Specify the namespace for the application.
    3 This is the Kopia password to connect to the repository.
    4 Specify the AWS S3 access key.
    5 Specify the AWS S3 storage provider secret access key.

    If you are using a storage provider other than AWS S3, you will need to add --endpoint, the bucket endpoint URL parameter, to the command.

  2. Verify that Kopia uses the environment variables that are configured in the DPA for the backup by running the following command:

    $ kopia repository status
    Example output
    Config file:         /../.config/kopia/repository.config

Description:         Repository in S3: s3.amazonaws.com <bucket_name>
# ...

Storage type:        s3
Storage capacity:    unbounded
Storage config:      {
                       "bucket": <bucket_name>,
                       "prefix": "velero/kopia/<application_namespace>/",
                       "endpoint": "s3.amazonaws.com",
                       "accessKeyID": <access_key>,
                       "secretAccessKey": "****************************************",
                       "sessionToken": ""
                     }

Unique ID:           58....aeb0
Hash:                BLAKE3-256
Encryption:          CHACHA20-POLY1305-HMAC-SHA256
Splitter:            DYNAMIC-8M-RABINKARP
Format version:      3
# ...

Benchmarking Kopia hashing, encryption, and splitter algorithms

You can run Kopia commands to benchmark the hashing, encryption, and splitter algorithms. Based on the benchmarking results, you can select the most suitable algorithm for your workload. In this procedure, you run the Kopia benchmarking commands from a pod on the cluster. The benchmarking results can vary depending on CPU speed, available RAM, disk speed, current I/O load, and so on.

Prerequisites

  • You have installed the OADP Operator.

  • You have an application with persistent volumes running in a separate namespace.

  • You have run a backup of the application with Container Storage Interface (CSI) snapshots.

The configuration of the Kopia algorithms for splitting, hashing, and encryption in the Data Protection Application (DPA) apply only during the initial Kopia repository creation, and cannot be changed later.

To use different Kopia algorithms, ensure that the object storage does not contain any previous Kopia repositories of backups. Configure a new object storage in the Backup Storage Location (BSL) or specify a unique prefix for the object storage in the BSL configuration.
Procedure

  1. Configure the must-gather pod as shown in the following example. Make sure you are using the oadp-mustgather image for OADP version 1.3 and later.

    Example pod configuration
    apiVersion: v1
kind: Pod
metadata:
  name: oadp-mustgather-pod
  labels:
    purpose: user-interaction
spec:
  containers:
  - name: oadp-mustgather-container
    image: registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.3
    command: ["sleep"]
    args: ["infinity"]

    The Kopia client is available in the oadp-mustgather image.

  2. Create the pod by running the following command:

    $ oc apply -f <pod_config_file_name> (1)
    1 Specify the name of the YAML file for the pod configuration.

  3. Verify that the Security Context Constraints (SCC) on the pod is anyuid, so that Kopia can connect to the repository.

    $ oc describe pod/oadp-mustgather-pod | grep scc
    Example output
    openshift.io/scc: anyuid

  4. Connect to the pod via SSH by running the following command:

    $ oc -n openshift-adp rsh pod/oadp-mustgather-pod

  5. Connect to the Kopia repository by running the following command:

    sh-5.1# kopia repository connect s3 \
  --bucket=<bucket_name> \ (1)
  --prefix=velero/kopia/<application_namespace> \ (2)
  --password=static-passw0rd \ (3)
  --access-key="<access_key>" \ (4)
  --secret-access-key="<secret_access_key>" \ (5)
  --endpoint=<bucket_endpoint> \ (6)
    1 Specify the object storage provider bucket name.
    2 Specify the namespace for the application.
    3 This is the Kopia password to connect to the repository.
    4 Specify the object storage provider access key.
    5 Specify the object storage provider secret access key.
    6 Specify the bucket endpoint. You do not need to specify the bucket endpoint, if you are using AWS S3 as the storage provider.

    This is an example command. The command can vary based on the object storage provider.

  6. To benchmark the hashing algorithm, run the following command:

    sh-5.1# kopia benchmark hashing
    Example output
    Benchmarking hash 'BLAKE2B-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2B-256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2S-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE2S-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE3-256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'BLAKE3-256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA224' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA256' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA256-128' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA3-224' (100 x 1048576 bytes, parallelism 1)
Benchmarking hash 'HMAC-SHA3-256' (100 x 1048576 bytes, parallelism 1)
     Hash                 Throughput
-----------------------------------------------------------------
  0. BLAKE3-256           15.3 GB / second
  1. BLAKE3-256-128       15.2 GB / second
  2. HMAC-SHA256-128      6.4 GB / second
  3. HMAC-SHA256          6.4 GB / second
  4. HMAC-SHA224          6.4 GB / second
  5. BLAKE2B-256-128      4.2 GB / second
  6. BLAKE2B-256          4.1 GB / second
  7. BLAKE2S-256          2.9 GB / second
  8. BLAKE2S-128          2.9 GB / second
  9. HMAC-SHA3-224        1.6 GB / second
 10. HMAC-SHA3-256        1.5 GB / second
-----------------------------------------------------------------
Fastest option for this machine is: --block-hash=BLAKE3-256

  7. To benchmark the encryption algorithm, run the following command:

    sh-5.1# kopia benchmark encryption
    Example output
    Benchmarking encryption 'AES256-GCM-HMAC-SHA256'... (1000 x 1048576 bytes, parallelism 1)
Benchmarking encryption 'CHACHA20-POLY1305-HMAC-SHA256'... (1000 x 1048576 bytes, parallelism 1)
     Encryption                     Throughput
-----------------------------------------------------------------
  0. AES256-GCM-HMAC-SHA256         2.2 GB / second
  1. CHACHA20-POLY1305-HMAC-SHA256  1.8 GB / second
-----------------------------------------------------------------
Fastest option for this machine is: --encryption=AES256-GCM-HMAC-SHA256

  8. To benchmark the splitter algorithm, run the following command:

    sh-5.1# kopia benchmark splitter
    Example output
    splitting 16 blocks of 32MiB each, parallelism 1
DYNAMIC                     747.6 MB/s count:107 min:9467 10th:2277562 25th:2971794 50th:4747177 75th:7603998 90th:8388608 max:8388608
DYNAMIC-128K-BUZHASH        718.5 MB/s count:3183 min:3076 10th:80896 25th:104312 50th:157621 75th:249115 90th:262144 max:262144
DYNAMIC-128K-RABINKARP      164.4 MB/s count:3160 min:9667 10th:80098 25th:106626 50th:162269 75th:250655 90th:262144 max:262144
# ...
FIXED-512K                  102.9 TB/s count:1024 min:524288 10th:524288 25th:524288 50th:524288 75th:524288 90th:524288 max:524288
FIXED-8M                    566.3 TB/s count:64 min:8388608 10th:8388608 25th:8388608 50th:8388608 75th:8388608 90th:8388608 max:8388608
-----------------------------------------------------------------
  0. FIXED-8M                  566.3 TB/s   count:64 min:8388608 10th:8388608 25th:8388608 50th:8388608 75th:8388608 90th:8388608 max:8388608
  1. FIXED-4M                  425.8 TB/s   count:128 min:4194304 10th:4194304 25th:4194304 50th:4194304 75th:4194304 90th:4194304 max:4194304
  # ...
 22. DYNAMIC-128K-RABINKARP    164.4 MB/s   count:3160 min:9667 10th:80098 25th:106626 50th:162269 75th:250655 90th:262144 max:262144