×

Some applications need sensitive information, such as passwords and user names, that you do not want developers to have.

As an alternative to using Kubernetes Secret objects to provide sensitive information, you can use an external secrets store to store the sensitive information. You can use the Secrets Store CSI Driver Operator to integrate with an external secrets store and mount the secret content as a pod volume.

The Secrets Store CSI Driver Operator is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

About the Secrets Store CSI Driver Operator

Kubernetes secrets are stored with Base64 encoding. etcd provides encryption at rest for these secrets, but when secrets are retrieved, they are decrypted and presented to the user. If role-based access control is not configured properly on your cluster, anyone with API or etcd access can retrieve or modify a secret. Additionally, anyone who is authorized to create a pod in a namespace can use that access to read any secret in that namespace.

To store and manage your secrets securely, you can configure the OKD Secrets Store Container Storage Interface (CSI) Driver Operator to mount secrets from an external secret management system, such as Azure Key Vault, by using a provider plugin. Applications can then use the secret, but the secret does not persist on the system after the application pod is destroyed.

The Secrets Store CSI Driver Operator, secrets-store.csi.k8s.io, enables OKD to mount multiple secrets, keys, and certificates stored in enterprise-grade external secrets stores into pods as a volume. The Secrets Store CSI Driver Operator communicates with the provider using gRPC to fetch the mount contents from the specified external secrets store. After the volume is attached, the data in it is mounted into the container’s file system. Secrets store volumes are mounted in-line.

Secrets store providers

The following secrets store providers are available for use with the Secrets Store CSI Driver Operator:

  • AWS Secrets Manager

  • AWS Systems Manager Parameter Store

  • Azure Key Vault

  • Google Secret Manager

  • HashiCorp Vault

Automatic rotation

The Secrets Store CSI driver periodically rotates the content in the mounted volume with the content from the external secrets store. If a secret is updated in the external secrets store, the secret will be updated in the mounted volume. The Secrets Store CSI Driver Operator polls for updates every 2 minutes.

If you enabled synchronization of mounted content as Kubernetes secrets, the Kubernetes secrets are also rotated.

Applications consuming the secret data must watch for updates to the secrets.

Installing the Secrets Store CSI driver

Prerequisites
  • Access to the OKD web console.

  • Administrator access to the cluster.

Procedure

To install the Secrets Store CSI driver:

  1. Install the Secrets Store CSI Driver Operator:

    1. Log in to the web console.

    2. Click OperatorsOperatorHub.

    3. Locate the Secrets Store CSI Driver Operator by typing "Secrets Store CSI" in the filter box.

    4. Click the Secrets Store CSI Driver Operator button.

    5. On the Secrets Store CSI Driver Operator page, click Install.

    6. On the Install Operator page, ensure that:

      • All namespaces on the cluster (default) is selected.

      • Installed Namespace is set to openshift-cluster-csi-drivers.

    7. Click Install.

      After the installation finishes, the Secrets Store CSI Driver Operator is listed in the Installed Operators section of the web console.

  2. Create the ClusterCSIDriver instance for the driver (secrets-store.csi.k8s.io):

    1. Click AdministrationCustomResourceDefinitionsClusterCSIDriver.

    2. On the Instances tab, click Create ClusterCSIDriver.

      Use the following YAML file:

      apiVersion: operator.openshift.io/v1
      kind: ClusterCSIDriver
      metadata:
          name: secrets-store.csi.k8s.io
      spec:
        managementState: Managed
    3. Click Create.

Mounting secrets from an external secrets store to a CSI volume

After installing the Secrets Store CSI Driver Operator, you can mount secrets from one of the following external secrets stores to a CSI volume:

Mounting secrets from AWS Secrets Manager

You can use the Secrets Store CSI Driver Operator to mount secrets from AWS Secrets Manager to a Container Storage Interface (CSI) volume in OKD. To mount secrets from AWS Secrets Manager, your cluster must be installed on AWS and use AWS Security Token Service (STS).

Prerequisites
  • Your cluster is installed on AWS and uses AWS Security Token Service (STS).

  • You installed the Secrets Store CSI Driver Operator. See Installing the Secrets Store CSI driver for instructions.

  • You configured AWS Secrets Manager to store the required secrets.

  • You extracted and prepared the ccoctl binary.

  • You installed the jq CLI tool.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Install the AWS Secrets Manager provider:

    1. Create a YAML file with the following configuration for the provider resources:

      The AWS Secrets Manager provider for the Secrets Store CSI driver is an upstream provider.

      This configuration is modified from the configuration provided in the upstream AWS documentation so that it works properly with OKD. Changes to this configuration might impact functionality.

      Example aws-provider.yaml file
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: csi-secrets-store-provider-aws
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: csi-secrets-store-provider-aws-cluster-role
      rules:
      - apiGroups: [""]
        resources: ["serviceaccounts/token"]
        verbs: ["create"]
      - apiGroups: [""]
        resources: ["serviceaccounts"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["nodes"]
        verbs: ["get"]
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: csi-secrets-store-provider-aws-cluster-rolebinding
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: csi-secrets-store-provider-aws-cluster-role
      subjects:
      - kind: ServiceAccount
        name: csi-secrets-store-provider-aws
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        namespace: openshift-cluster-csi-drivers
        name: csi-secrets-store-provider-aws
        labels:
          app: csi-secrets-store-provider-aws
      spec:
        updateStrategy:
          type: RollingUpdate
        selector:
          matchLabels:
            app: csi-secrets-store-provider-aws
        template:
          metadata:
            labels:
              app: csi-secrets-store-provider-aws
          spec:
            serviceAccountName: csi-secrets-store-provider-aws
            hostNetwork: false
            containers:
              - name: provider-aws-installer
                image: public.ecr.aws/aws-secrets-manager/secrets-store-csi-driver-provider-aws:1.0.r2-50-g5b4aca1-2023.06.09.21.19
                imagePullPolicy: Always
                args:
                    - --provider-volume=/etc/kubernetes/secrets-store-csi-providers
                resources:
                  requests:
                    cpu: 50m
                    memory: 100Mi
                  limits:
                    cpu: 50m
                    memory: 100Mi
                securityContext:
                  privileged: true
                volumeMounts:
                  - mountPath: "/etc/kubernetes/secrets-store-csi-providers"
                    name: providervol
                  - name: mountpoint-dir
                    mountPath: /var/lib/kubelet/pods
                    mountPropagation: HostToContainer
            tolerations:
            - operator: Exists
            volumes:
              - name: providervol
                hostPath:
                  path: "/etc/kubernetes/secrets-store-csi-providers"
              - name: mountpoint-dir
                hostPath:
                  path: /var/lib/kubelet/pods
                  type: DirectoryOrCreate
            nodeSelector:
              kubernetes.io/os: linux
    2. Grant privileged access to the csi-secrets-store-provider-aws service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z csi-secrets-store-provider-aws -n openshift-cluster-csi-drivers
    3. Create the provider resources by running the following command:

      $ oc apply -f aws-provider.yaml
  2. Grant permission to allow the service account to read the AWS secret object:

    1. Create a directory to contain the credentials request by running the following command:

      $ mkdir credentialsrequest-dir-aws
    2. Create a YAML file with the following configuration for the credentials request:

      Example credentialsrequest.yaml file
      apiVersion: cloudcredential.openshift.io/v1
      kind: CredentialsRequest
      metadata:
        name: aws-provider-test
        namespace: openshift-cloud-credential-operator
      spec:
        providerSpec:
          apiVersion: cloudcredential.openshift.io/v1
          kind: AWSProviderSpec
          statementEntries:
          - action:
            - "secretsmanager:GetSecretValue"
            - "secretsmanager:DescribeSecret"
            effect: Allow
            resource: "arn:*:secretsmanager:*:*:secret:testSecret-??????"
        secretRef:
          name: aws-creds
          namespace: my-namespace
        serviceAccountNames:
        - aws-provider
    3. Retrieve the OIDC provider by running the following command:

      $ oc get --raw=/.well-known/openid-configuration | jq -r '.issuer'
      Example output
      https://<oidc_provider_name>

      Copy the OIDC provider name <oidc_provider_name> from the output to use in the next step.

    4. Use the ccoctl tool to process the credentials request by running the following command:

      $ ccoctl aws create-iam-roles \
          --name my-role --region=<aws_region> \
          --credentials-requests-dir=credentialsrequest-dir-aws \
          --identity-provider-arn arn:aws:iam::<aws_account>:oidc-provider/<oidc_provider_name> --output-dir=credrequests-ccoctl-output
      Example output
      2023/05/15 18:10:34 Role arn:aws:iam::<aws_account_id>:role/my-role-my-namespace-aws-creds created
      2023/05/15 18:10:34 Saved credentials configuration to: credrequests-ccoctl-output/manifests/my-namespace-aws-creds-credentials.yaml
      2023/05/15 18:10:35 Updated Role policy for Role my-role-my-namespace-aws-creds

      Copy the <aws_role_arn> from the output to use in the next step. For example, arn:aws:iam::<aws_account_id>:role/my-role-my-namespace-aws-creds.

    5. Bind the service account with the role ARN by running the following command:

      $ oc annotate -n my-namespace sa/aws-provider eks.amazonaws.com/role-arn="<aws_role_arn>"
  3. Create a secret provider class to define your secrets store provider:

    1. Create a YAML file that defines the SecretProviderClass object:

      Example secret-provider-class-aws.yaml
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: my-aws-provider                   (1)
        namespace: my-namespace                 (2)
      spec:
        provider: aws                           (3)
        parameters:                             (4)
          objects: |
            - objectName: "testSecret"
              objectType: "secretsmanager"
      1 Specify the name for the secret provider class.
      2 Specify the namespace for the secret provider class.
      3 Specify the provider as aws.
      4 Specify the provider-specific configuration parameters.
    2. Create the SecretProviderClass object by running the following command:

      $ oc create -f secret-provider-class-aws.yaml
  4. Create a deployment to use this secret provider class:

    1. Create a YAML file that defines the Deployment object:

      Example deployment.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: my-aws-deployment                              (1)
        namespace: my-namespace                              (2)
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: my-storage
        template:
          metadata:
            labels:
              app: my-storage
          spec:
            serviceAccountName: aws-provider
            containers:
            - name: busybox
              image: k8s.gcr.io/e2e-test-images/busybox:1.29
              command:
                - "/bin/sleep"
                - "10000"
              volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true
            volumes:
              - name: secrets-store-inline
                csi:
                  driver: secrets-store.csi.k8s.io
                  readOnly: true
                  volumeAttributes:
                    secretProviderClass: "my-aws-provider" (3)
      1 Specify the name for the deployment.
      2 Specify the namespace for the deployment. This must be the same namespace as the secret provider class.
      3 Specify the name of the secret provider class.
    2. Create the Deployment object by running the following command:

      $ oc create -f deployment.yaml
Verification
  • Verify that you can access the secrets from AWS Secrets Manager in the pod volume mount:

    1. List the secrets in the pod mount by running the following command:

      $ oc exec my-aws-deployment-<hash> -n my-namespace -- ls /mnt/secrets-store/
      Example output
      testSecret
    2. View a secret in the pod mount by running the following command:

      $ oc exec my-aws-deployment-<hash> -n my-namespace -- cat /mnt/secrets-store/testSecret
      Example output
      <secret_value>

Mounting secrets from AWS Systems Manager Parameter Store

You can use the Secrets Store CSI Driver Operator to mount secrets from AWS Systems Manager Parameter Store to a Container Storage Interface (CSI) volume in OKD. To mount secrets from AWS Systems Manager Parameter Store, your cluster must be installed on AWS and use AWS Security Token Service (STS).

Prerequisites
  • Your cluster is installed on AWS and uses AWS Security Token Service (STS).

  • You installed the Secrets Store CSI Driver Operator. See Installing the Secrets Store CSI driver for instructions.

  • You configured AWS Systems Manager Parameter Store to store the required secrets.

  • You extracted and prepared the ccoctl binary.

  • You installed the jq CLI tool.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Install the AWS Systems Manager Parameter Store provider:

    1. Create a YAML file with the following configuration for the provider resources:

      The AWS Systems Manager Parameter Store provider for the Secrets Store CSI driver is an upstream provider.

      This configuration is modified from the configuration provided in the upstream AWS documentation so that it works properly with OKD. Changes to this configuration might impact functionality.

      Example aws-provider.yaml file
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: csi-secrets-store-provider-aws
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: csi-secrets-store-provider-aws-cluster-role
      rules:
      - apiGroups: [""]
        resources: ["serviceaccounts/token"]
        verbs: ["create"]
      - apiGroups: [""]
        resources: ["serviceaccounts"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["nodes"]
        verbs: ["get"]
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: csi-secrets-store-provider-aws-cluster-rolebinding
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: csi-secrets-store-provider-aws-cluster-role
      subjects:
      - kind: ServiceAccount
        name: csi-secrets-store-provider-aws
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        namespace: openshift-cluster-csi-drivers
        name: csi-secrets-store-provider-aws
        labels:
          app: csi-secrets-store-provider-aws
      spec:
        updateStrategy:
          type: RollingUpdate
        selector:
          matchLabels:
            app: csi-secrets-store-provider-aws
        template:
          metadata:
            labels:
              app: csi-secrets-store-provider-aws
          spec:
            serviceAccountName: csi-secrets-store-provider-aws
            hostNetwork: false
            containers:
              - name: provider-aws-installer
                image: public.ecr.aws/aws-secrets-manager/secrets-store-csi-driver-provider-aws:1.0.r2-50-g5b4aca1-2023.06.09.21.19
                imagePullPolicy: Always
                args:
                    - --provider-volume=/etc/kubernetes/secrets-store-csi-providers
                resources:
                  requests:
                    cpu: 50m
                    memory: 100Mi
                  limits:
                    cpu: 50m
                    memory: 100Mi
                securityContext:
                  privileged: true
                volumeMounts:
                  - mountPath: "/etc/kubernetes/secrets-store-csi-providers"
                    name: providervol
                  - name: mountpoint-dir
                    mountPath: /var/lib/kubelet/pods
                    mountPropagation: HostToContainer
            tolerations:
            - operator: Exists
            volumes:
              - name: providervol
                hostPath:
                  path: "/etc/kubernetes/secrets-store-csi-providers"
              - name: mountpoint-dir
                hostPath:
                  path: /var/lib/kubelet/pods
                  type: DirectoryOrCreate
            nodeSelector:
              kubernetes.io/os: linux
    2. Grant privileged access to the csi-secrets-store-provider-aws service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z csi-secrets-store-provider-aws -n openshift-cluster-csi-drivers
    3. Create the provider resources by running the following command:

      $ oc apply -f aws-provider.yaml
  2. Grant permission to allow the service account to read the AWS secret object:

    1. Create a directory to contain the credentials request by running the following command:

      $ mkdir credentialsrequest-dir-aws
    2. Create a YAML file with the following configuration for the credentials request:

      Example credentialsrequest.yaml file
      apiVersion: cloudcredential.openshift.io/v1
      kind: CredentialsRequest
      metadata:
        name: aws-provider-test
        namespace: openshift-cloud-credential-operator
      spec:
        providerSpec:
          apiVersion: cloudcredential.openshift.io/v1
          kind: AWSProviderSpec
          statementEntries:
          - action:
            - "ssm:GetParameter"
            - "ssm:GetParameters"
            effect: Allow
            resource: "arn:*:ssm:*:*:parameter/testParameter*"
        secretRef:
          name: aws-creds
          namespace: my-namespace
        serviceAccountNames:
        - aws-provider
    3. Retrieve the OIDC provider by running the following command:

      $ oc get --raw=/.well-known/openid-configuration | jq -r '.issuer'
      Example output
      https://<oidc_provider_name>

      Copy the OIDC provider name <oidc_provider_name> from the output to use in the next step.

    4. Use the ccoctl tool to process the credentials request by running the following command:

      $ ccoctl aws create-iam-roles \
          --name my-role --region=<aws_region> \
          --credentials-requests-dir=credentialsrequest-dir-aws \
          --identity-provider-arn arn:aws:iam::<aws_account>:oidc-provider/<oidc_provider_name> --output-dir=credrequests-ccoctl-output
      Example output
      2023/05/15 18:10:34 Role arn:aws:iam::<aws_account_id>:role/my-role-my-namespace-aws-creds created
      2023/05/15 18:10:34 Saved credentials configuration to: credrequests-ccoctl-output/manifests/my-namespace-aws-creds-credentials.yaml
      2023/05/15 18:10:35 Updated Role policy for Role my-role-my-namespace-aws-creds

      Copy the <aws_role_arn> from the output to use in the next step. For example, arn:aws:iam::<aws_account_id>:role/my-role-my-namespace-aws-creds.

    5. Bind the service account with the role ARN by running the following command:

      $ oc annotate -n my-namespace sa/aws-provider eks.amazonaws.com/role-arn="<aws_role_arn>"
  3. Create a secret provider class to define your secrets store provider:

    1. Create a YAML file that defines the SecretProviderClass object:

      Example secret-provider-class-aws.yaml
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: my-aws-provider                   (1)
        namespace: my-namespace                 (2)
      spec:
        provider: aws                           (3)
        parameters:                             (4)
          objects: |
            - objectName: "testParameter"
              objectType: "ssmparameter"
      1 Specify the name for the secret provider class.
      2 Specify the namespace for the secret provider class.
      3 Specify the provider as aws.
      4 Specify the provider-specific configuration parameters.
    2. Create the SecretProviderClass object by running the following command:

      $ oc create -f secret-provider-class-aws.yaml
  4. Create a deployment to use this secret provider class:

    1. Create a YAML file that defines the Deployment object:

      Example deployment.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: my-aws-deployment                              (1)
        namespace: my-namespace                              (2)
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: my-storage
        template:
          metadata:
            labels:
              app: my-storage
          spec:
            serviceAccountName: aws-provider
            containers:
            - name: busybox
              image: k8s.gcr.io/e2e-test-images/busybox:1.29
              command:
                - "/bin/sleep"
                - "10000"
              volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true
            volumes:
              - name: secrets-store-inline
                csi:
                  driver: secrets-store.csi.k8s.io
                  readOnly: true
                  volumeAttributes:
                    secretProviderClass: "my-aws-provider" (3)
      1 Specify the name for the deployment.
      2 Specify the namespace for the deployment. This must be the same namespace as the secret provider class.
      3 Specify the name of the secret provider class.
    2. Create the Deployment object by running the following command:

      $ oc create -f deployment.yaml
Verification
  • Verify that you can access the secrets from AWS Systems Manager Parameter Store in the pod volume mount:

    1. List the secrets in the pod mount by running the following command:

      $ oc exec my-aws-deployment-<hash> -n my-namespace -- ls /mnt/secrets-store/
      Example output
      testParameter
    2. View a secret in the pod mount by running the following command:

      $ oc exec my-aws-deployment-<hash> -n my-namespace -- cat /mnt/secrets-store/testSecret
      Example output
      <secret_value>

Mounting secrets from Azure Key Vault

You can use the Secrets Store CSI Driver Operator to mount secrets from Azure Key Vault to a Container Storage Interface (CSI) volume in OKD. To mount secrets from Azure Key Vault, your cluster must be installed on Microsoft Azure.

Prerequisites
  • Your cluster is installed on Azure.

  • You installed the Secrets Store CSI Driver Operator. See Installing the Secrets Store CSI driver for instructions.

  • You configured Azure Key Vault to store the required secrets.

  • You installed the Azure CLI (az).

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Install the Azure Key Vault provider:

    1. Create a YAML file with the following configuration for the provider resources:

      The Azure Key Vault provider for the Secrets Store CSI driver is an upstream provider.

      This configuration is modified from the configuration provided in the upstream Azure documentation so that it works properly with OKD. Changes to this configuration might impact functionality.

      Example azure-provider.yaml file
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: csi-secrets-store-provider-azure
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: csi-secrets-store-provider-azure-cluster-role
      rules:
      - apiGroups: [""]
        resources: ["serviceaccounts/token"]
        verbs: ["create"]
      - apiGroups: [""]
        resources: ["serviceaccounts"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["pods"]
        verbs: ["get"]
      - apiGroups: [""]
        resources: ["nodes"]
        verbs: ["get"]
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: csi-secrets-store-provider-azure-cluster-rolebinding
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: csi-secrets-store-provider-azure-cluster-role
      subjects:
      - kind: ServiceAccount
        name: csi-secrets-store-provider-azure
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        namespace: openshift-cluster-csi-drivers
        name: csi-secrets-store-provider-azure
        labels:
          app: csi-secrets-store-provider-azure
      spec:
        updateStrategy:
          type: RollingUpdate
        selector:
          matchLabels:
            app: csi-secrets-store-provider-azure
        template:
          metadata:
            labels:
              app: csi-secrets-store-provider-azure
          spec:
            serviceAccountName: csi-secrets-store-provider-azure
            hostNetwork: true
            containers:
              - name: provider-azure-installer
                image: mcr.microsoft.com/oss/azure/secrets-store/provider-azure:v1.4.1
                imagePullPolicy: IfNotPresent
                args:
                  - --endpoint=unix:///provider/azure.sock
                  - --construct-pem-chain=true
                  - --healthz-port=8989
                  - --healthz-path=/healthz
                  - --healthz-timeout=5s
                livenessProbe:
                  httpGet:
                    path: /healthz
                    port: 8989
                  failureThreshold: 3
                  initialDelaySeconds: 5
                  timeoutSeconds: 10
                  periodSeconds: 30
                resources:
                  requests:
                    cpu: 50m
                    memory: 100Mi
                  limits:
                    cpu: 50m
                    memory: 100Mi
                securityContext:
                  allowPrivilegeEscalation: false
                  readOnlyRootFilesystem: true
                  runAsUser: 0
                  capabilities:
                    drop:
                    - ALL
                volumeMounts:
                  - mountPath: "/provider"
                    name: providervol
            affinity:
              nodeAffinity:
                requiredDuringSchedulingIgnoredDuringExecution:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: type
                      operator: NotIn
                      values:
                      - virtual-kubelet
            volumes:
              - name: providervol
                hostPath:
                  path: "/var/run/secrets-store-csi-providers"
            tolerations:
            - operator: Exists
            nodeSelector:
              kubernetes.io/os: linux
    2. Grant privileged access to the csi-secrets-store-provider-azure service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z csi-secrets-store-provider-azure -n openshift-cluster-csi-drivers
    3. Create the provider resources by running the following command:

      $ oc apply -f azure-provider.yaml
  2. Create a service principal to access the key vault:

    1. Set the service principal client secret as an environment variable by running the following command:

      $ SERVICE_PRINCIPAL_CLIENT_SECRET="$(az ad sp create-for-rbac --name https://$KEYVAULT_NAME --query 'password' -otsv)"
    2. Set the service principal client ID as an environment variable by running the following command:

      $ SERVICE_PRINCIPAL_CLIENT_ID="$(az ad sp list --display-name https://$KEYVAULT_NAME --query '[0].appId' -otsv)"
    3. Create a generic secret with the service principal client secret and ID by running the following command:

      $ oc create secret generic secrets-store-creds -n my-namespace --from-literal clientid=${SERVICE_PRINCIPAL_CLIENT_ID} --from-literal clientsecret=${SERVICE_PRINCIPAL_CLIENT_SECRET}
    4. Apply the secrets-store.csi.k8s.io/used=true label to allow the provider to find this nodePublishSecretRef secret:

      $ oc -n my-namespace label secret secrets-store-creds secrets-store.csi.k8s.io/used=true
  3. Create a secret provider class to define your secrets store provider:

    1. Create a YAML file that defines the SecretProviderClass object:

      Example secret-provider-class-azure.yaml
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: my-azure-provider                 (1)
        namespace: my-namespace                 (2)
      spec:
        provider: azure                         (3)
        parameters:                             (4)
          usePodIdentity: "false"
          useVMManagedIdentity: "false"
          userAssignedIdentityID: ""
          keyvaultName: "kvname"
          objects: |
            array:
              - |
                objectName: secret1
                objectType: secret
          tenantId: "tid"
      1 Specify the name for the secret provider class.
      2 Specify the namespace for the secret provider class.
      3 Specify the provider as azure.
      4 Specify the provider-specific configuration parameters.
    2. Create the SecretProviderClass object by running the following command:

      $ oc create -f secret-provider-class-azure.yaml
  4. Create a deployment to use this secret provider class:

    1. Create a YAML file that defines the Deployment object:

      Example deployment.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: my-azure-deployment                            (1)
        namespace: my-namespace                              (2)
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: my-storage
        template:
          metadata:
            labels:
              app: my-storage
          spec:
            containers:
            - name: busybox
              image: k8s.gcr.io/e2e-test-images/busybox:1.29
              command:
                - "/bin/sleep"
                - "10000"
              volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true
            volumes:
              - name: secrets-store-inline
                csi:
                  driver: secrets-store.csi.k8s.io
                  readOnly: true
                  volumeAttributes:
                    secretProviderClass: "my-azure-provider" (3)
                  nodePublishSecretRef:
                    name: secrets-store-creds                (4)
      1 Specify the name for the deployment.
      2 Specify the namespace for the deployment. This must be the same namespace as the secret provider class.
      3 Specify the name of the secret provider class.
      4 Specify the name of the Kubernetes secret that contains the service principal credentials to access Azure Key Vault.
    2. Create the Deployment object by running the following command:

      $ oc create -f deployment.yaml
Verification
  • Verify that you can access the secrets from Azure Key Vault in the pod volume mount:

    1. List the secrets in the pod mount by running the following command:

      $ oc exec my-azure-deployment-<hash> -n my-namespace -- ls /mnt/secrets-store/
      Example output
      secret1
    2. View a secret in the pod mount by running the following command:

      $ oc exec my-azure-deployment-<hash> -n my-namespace -- cat /mnt/secrets-store/secret1
      Example output
      my-secret-value

Mounting secrets from Google Secret Manager

You can use the Secrets Store CSI Driver Operator to mount secrets from Google Secret Manager to a Container Storage Interface (CSI) volume in OKD. To mount secrets from Google Secret Manager, your cluster must be installed on Google Cloud Platform (GCP).

Prerequisites
  • You installed the Secrets Store CSI Driver Operator. See Installing the Secrets Store CSI driver for instructions.

  • You configured Google Secret Manager to store the required secrets.

  • You created a service account key named key.json from your Google Cloud service account.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Install the Google Secret Manager provider:

    1. Create a YAML file with the following configuration for the provider resources:

      Example gcp-provider.yaml file
      apiVersion: v1
      kind: ServiceAccount
      metadata:
        name: csi-secrets-store-provider-gcp
        namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRoleBinding
      metadata:
        name: csi-secrets-store-provider-gcp-rolebinding
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: csi-secrets-store-provider-gcp-role
      subjects:
        - kind: ServiceAccount
          name: csi-secrets-store-provider-gcp
          namespace: openshift-cluster-csi-drivers
      ---
      apiVersion: rbac.authorization.k8s.io/v1
      kind: ClusterRole
      metadata:
        name: csi-secrets-store-provider-gcp-role
      rules:
        - apiGroups:
            - ""
          resources:
            - serviceaccounts/token
          verbs:
            - create
        - apiGroups:
            - ""
          resources:
            - serviceaccounts
          verbs:
            - get
      ---
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: csi-secrets-store-provider-gcp
        namespace: openshift-cluster-csi-drivers
        labels:
          app: csi-secrets-store-provider-gcp
      spec:
        updateStrategy:
          type: RollingUpdate
        selector:
          matchLabels:
            app: csi-secrets-store-provider-gcp
        template:
          metadata:
            labels:
              app: csi-secrets-store-provider-gcp
          spec:
            serviceAccountName: csi-secrets-store-provider-gcp
            initContainers:
            - name: chown-provider-mount
              image: busybox
              command:
              - chown
              - "1000:1000"
              - /etc/kubernetes/secrets-store-csi-providers
              volumeMounts:
              - mountPath: "/etc/kubernetes/secrets-store-csi-providers"
                name: providervol
              securityContext:
                privileged: true
            hostNetwork: false
            hostPID: false
            hostIPC: false
            containers:
              - name: provider
                image: us-docker.pkg.dev/secretmanager-csi/secrets-store-csi-driver-provider-gcp/plugin@sha256:a493a78bbb4ebce5f5de15acdccc6f4d19486eae9aa4fa529bb60ac112dd6650
                securityContext:
                  privileged: true
                imagePullPolicy: IfNotPresent
                resources:
                  requests:
                    cpu: 50m
                    memory: 100Mi
                  limits:
                    cpu: 50m
                    memory: 100Mi
                env:
                  - name: TARGET_DIR
                    value: "/etc/kubernetes/secrets-store-csi-providers"
                volumeMounts:
                  - mountPath: "/etc/kubernetes/secrets-store-csi-providers"
                    name: providervol
                    mountPropagation: None
                    readOnly: false
                livenessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /live
                    port: 8095
                  initialDelaySeconds: 5
                  timeoutSeconds: 10
                  periodSeconds: 30
            volumes:
              - name: providervol
                hostPath:
                  path: /etc/kubernetes/secrets-store-csi-providers
            tolerations:
              - key: kubernetes.io/arch
                operator: Equal
                value: amd64
                effect: NoSchedule
            nodeSelector:
              kubernetes.io/os: linux
    2. Grant privileged access to the csi-secrets-store-provider-gcp service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z csi-secrets-store-provider-gcp -n openshift-cluster-csi-drivers
    3. Create the provider resources by running the following command:

      $ oc apply -f gcp-provider.yaml
  2. Grant permission to read the Google Secret Manager secret:

    1. Create a new project by running the following command:

      $ oc new-project my-namespace
    2. Label the my-namespace namespace for pod security admission by running the following command:

      $ oc label ns my-namespace security.openshift.io/scc.podSecurityLabelSync=false pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/audit=privileged pod-security.kubernetes.io/warn=privileged --overwrite
    3. Create a service account for the pod deployment:

      $ oc create serviceaccount my-service-account --namespace=my-namespace
    4. Create a generic secret from the key.json file by running the following command:

      $ oc create secret generic secrets-store-creds -n my-namespace --from-file=key.json (1)
      1 You created this key.json file from the Google Secret Manager.
    5. Apply the secrets-store.csi.k8s.io/used=true label to allow the provider to find this nodePublishSecretRef secret:

      $ oc -n my-namespace label secret secrets-store-creds secrets-store.csi.k8s.io/used=true
  3. Create a secret provider class to define your secrets store provider:

    1. Create a YAML file that defines the SecretProviderClass object:

      Example secret-provider-class-gcp.yaml
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: my-gcp-provider                        (1)
        namespace: my-namespace                      (2)
      spec:
        provider: gcp                                (3)
        parameters:                                  (4)
          secrets: |
            - resourceName: "projects/my-project/secrets/testsecret1/versions/1"
              path: "testsecret1.txt"
      1 Specify the name for the secret provider class.
      2 Specify the namespace for the secret provider class.
      3 Specify the provider as gcp.
      4 Specify the provider-specific configuration parameters.
    2. Create the SecretProviderClass object by running the following command:

      $ oc create -f secret-provider-class-gcp.yaml
  4. Create a deployment to use this secret provider class:

    1. Create a YAML file that defines the Deployment object:

      Example deployment.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: my-gcp-deployment                              (1)
        namespace: my-namespace                              (2)
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: my-storage
        template:
          metadata:
            labels:
              app: my-storage
          spec:
            serviceAccountName: my-service-account           (3)
            containers:
            - name: busybox
              image: k8s.gcr.io/e2e-test-images/busybox:1.29
              command:
                - "/bin/sleep"
                - "10000"
              volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true
            volumes:
              - name: secrets-store-inline
                csi:
                  driver: secrets-store.csi.k8s.io
                  readOnly: true
                  volumeAttributes:
                    secretProviderClass: "my-gcp-provider"   (4)
                  nodePublishSecretRef:
                    name: secrets-store-creds                (5)
      1 Specify the name for the deployment.
      2 Specify the namespace for the deployment. This must be the same namespace as the secret provider class.
      3 Specify the service account you created.
      4 Specify the name of the secret provider class.
      5 Specify the name of the Kubernetes secret that contains the service principal credentials to access Google Secret Manager.
    2. Create the Deployment object by running the following command:

      $ oc create -f deployment.yaml
Verification
  • Verify that you can access the secrets from Google Secret Manager in the pod volume mount:

    1. List the secrets in the pod mount by running the following command:

      $ oc exec my-gcp-deployment-<hash> -n my-namespace -- ls /mnt/secrets-store/
      Example output
      testsecret1
    2. View a secret in the pod mount by running the following command:

      $ oc exec my-gcp-deployment-<hash> -n my-namespace -- cat /mnt/secrets-store/testsecret1
      Example output
      <secret_value>

Mounting secrets from HashiCorp Vault

You can use the Secrets Store CSI Driver Operator to mount secrets from HashiCorp Vault to a Container Storage Interface (CSI) volume in OKD.

Mounting secrets from HashiCorp Vault by using the Secrets Store CSI Driver Operator has been tested with the following cloud providers:

  • Amazon Web Services (AWS)

  • Microsoft Azure

Other cloud providers might work, but have not been tested yet. Additional cloud providers might be tested in the future.

Prerequisites
  • You installed the Secrets Store CSI Driver Operator. See Installing the Secrets Store CSI driver for instructions.

  • You installed Helm.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Add the HashiCorp Helm repository by running the following command:

    $ helm repo add hashicorp https://helm.releases.hashicorp.com
  2. Update all repositories to ensure that Helm is aware of the latest versions by running the following command:

    $ helm repo update
  3. Install the HashiCorp Vault provider:

    1. Create a new project for Vault by running the following command:

      $ oc new-project vault
    2. Label the vault namespace for pod security admission by running the following command:

      $ oc label ns vault security.openshift.io/scc.podSecurityLabelSync=false pod-security.kubernetes.io/enforce=privileged pod-security.kubernetes.io/audit=privileged pod-security.kubernetes.io/warn=privileged --overwrite
    3. Grant privileged access to the vault service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z vault -n vault
    4. Grant privileged access to the vault-csi-provider service account by running the following command:

      $ oc adm policy add-scc-to-user privileged -z vault-csi-provider -n vault
    5. Deploy HashiCorp Vault by running the following command:

      $ helm install vault hashicorp/vault --namespace=vault \
        --set "server.dev.enabled=true" \
        --set "injector.enabled=false" \
        --set "csi.enabled=true" \
        --set "global.openshift=true" \
        --set "injector.agentImage.repository=docker.io/hashicorp/vault" \
        --set "server.image.repository=docker.io/hashicorp/vault" \
        --set "csi.image.repository=docker.io/hashicorp/vault-csi-provider" \
        --set "csi.agent.image.repository=docker.io/hashicorp/vault" \
        --set "csi.daemonSet.providersDir=/var/run/secrets-store-csi-providers"
    6. Patch the vault-csi-driver daemon set to set the securityContext to privileged by running the following command:

      $ oc patch daemonset -n vault vault-csi-provider --type='json' -p='[{"op": "add", "path": "/spec/template/spec/containers/0/securityContext", "value": {"privileged": true} }]'
    7. Verify that the vault-csi-provider pods have started properly by running the following command:

      $ oc get pods -n vault
      Example output
      NAME                       READY   STATUS    RESTARTS   AGE
      vault-0                    1/1     Running   0          24m
      vault-csi-provider-87rgw   1/2     Running   0          5s
      vault-csi-provider-bd6hp   1/2     Running   0          4s
      vault-csi-provider-smlv7   1/2     Running   0          5s
  4. Configure HashiCorp Vault to store the required secrets:

    1. Create a secret by running the following command:

      $ oc exec vault-0 --namespace=vault -- vault kv put secret/example1 testSecret1=my-secret-value
    2. Verify that the secret is readable at the path secret/example1 by running the following command:

      $ oc exec vault-0 --namespace=vault -- vault kv get secret/example1
      Example output
      = Secret Path =
      secret/data/example1
      
      ======= Metadata =======
      Key                Value
      ---                -----
      created_time       2024-04-05T07:05:16.713911211Z
      custom_metadata    <nil>
      deletion_time      n/a
      destroyed          false
      version            1
      
      === Data ===
      Key            Value
      ---            -----
      testSecret1    my-secret-value
  5. Configure Vault to use Kubernetes authentication:

    1. Enable the Kubernetes auth method by running the following command:

      $ oc exec vault-0 --namespace=vault -- vault auth enable kubernetes
      Example output
      Success! Enabled kubernetes auth method at: kubernetes/
    2. Configure the Kubernetes auth method:

      1. Set the token reviewer as an environment variable by running the following command:

        $ TOKEN_REVIEWER_JWT="$(oc exec vault-0 --namespace=vault -- cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
      2. Set the Kubernetes service IP address as an environment variable by running the following command:

        $ KUBERNETES_SERVICE_IP="$(oc get svc kubernetes -o go-template="{{ .spec.clusterIP }}")"
      3. Update the Kubernetes auth method by running the following command:

        $ oc exec -i vault-0 --namespace=vault -- vault write auth/kubernetes/config \
          issuer="https://kubernetes.default.svc.cluster.local" \
          token_reviewer_jwt="${TOKEN_REVIEWER_JWT}" \
          kubernetes_host="https://${KUBERNETES_SERVICE_IP}:443" \
          kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        Example output
        Success! Data written to: auth/kubernetes/config
    3. Create a policy for the application by running the following command:

      $ oc exec -i vault-0 --namespace=vault -- vault policy write csi -<<EOF
        path "secret/data/*" {
        capabilities = ["read"]
        }
        EOF
      Example output
      Success! Uploaded policy: csi
    4. Create an authentication role to access the application by running the following command:

      $ oc exec -i vault-0 --namespace=vault -- vault write auth/kubernetes/role/csi \
        bound_service_account_names=default \
        bound_service_account_namespaces=default,test-ns,negative-test-ns,my-namespace \
        policies=csi \
        ttl=20m
      Example output
      Success! Data written to: auth/kubernetes/role/csi
    5. Verify that all of the vault pods are running properly by running the following command:

      $ oc get pods -n vault
      Example output
      NAME                       READY   STATUS    RESTARTS   AGE
      vault-0                    1/1     Running   0          43m
      vault-csi-provider-87rgw   2/2     Running   0          19m
      vault-csi-provider-bd6hp   2/2     Running   0          19m
      vault-csi-provider-smlv7   2/2     Running   0          19m
    6. Verify that all of the secrets-store-csi-driver pods are running properly by running the following command:

      $ oc get pods -n openshift-cluster-csi-drivers | grep -E "secrets"
      Example output
      secrets-store-csi-driver-node-46d2g                  3/3     Running   0             45m
      secrets-store-csi-driver-node-d2jjn                  3/3     Running   0             45m
      secrets-store-csi-driver-node-drmt4                  3/3     Running   0             45m
      secrets-store-csi-driver-node-j2wlt                  3/3     Running   0             45m
      secrets-store-csi-driver-node-v9xv4                  3/3     Running   0             45m
      secrets-store-csi-driver-node-vlz28                  3/3     Running   0             45m
      secrets-store-csi-driver-operator-84bd699478-fpxrw   1/1     Running   0             47m
  6. Create a secret provider class to define your secrets store provider:

    1. Create a YAML file that defines the SecretProviderClass object:

      Example secret-provider-class-vault.yaml
      apiVersion: secrets-store.csi.x-k8s.io/v1
      kind: SecretProviderClass
      metadata:
        name: my-vault-provider                   (1)
        namespace: my-namespace                   (2)
      spec:
        provider: vault                           (3)
        parameters:                               (4)
          roleName: "csi"
          vaultAddress: "http://vault.vault:8200"
          objects:  |
            - secretPath: "secret/data/example1"
              objectName: "testSecret1"
              secretKey: "testSecret1
      1 Specify the name for the secret provider class.
      2 Specify the namespace for the secret provider class.
      3 Specify the provider as vault.
      4 Specify the provider-specific configuration parameters.
    2. Create the SecretProviderClass object by running the following command:

      $ oc create -f secret-provider-class-vault.yaml
  7. Create a deployment to use this secret provider class:

    1. Create a YAML file that defines the Deployment object:

      Example deployment.yaml
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: busybox-deployment                                    (1)
        namespace: my-namespace                                     (2)
        labels:
          app: busybox
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: busybox
        template:
          metadata:
            labels:
              app: busybox
          spec:
            terminationGracePeriodSeconds: 0
            containers:
            - image: registry.k8s.io/e2e-test-images/busybox:1.29-4
              name: busybox
              imagePullPolicy: IfNotPresent
              command:
              - "/bin/sleep"
              - "10000"
              volumeMounts:
              - name: secrets-store-inline
                mountPath: "/mnt/secrets-store"
                readOnly: true
            volumes:
              - name: secrets-store-inline
                csi:
                  driver: secrets-store.csi.k8s.io
                  readOnly: true
                  volumeAttributes:
                    secretProviderClass: "my-vault-provider"        (3)
      1 Specify the name for the deployment.
      2 Specify the namespace for the deployment. This must be the same namespace as the secret provider class.
      3 Specify the name of the secret provider class.
    2. Create the Deployment object by running the following command:

      $ oc create -f deployment.yaml
Verification
  • Verify that you can access the secrets from your HashiCorp Vault in the pod volume mount:

    1. List the secrets in the pod mount by running the following command:

      $ oc exec busybox-deployment-<hash> -n my-namespace -- ls /mnt/secrets-store/
      Example output
      testSecret1
    2. View a secret in the pod mount by running the following command:

      $ oc exec busybox-deployment-<hash> -n my-namespace -- cat /mnt/secrets-store/testSecret1
      Example output
      my-secret-value

Enabling synchronization of mounted content as Kubernetes secrets

You can enable synchronization to create Kubernetes secrets from the content on a mounted volume. An example where you might want to enable synchronization is to use an environment variable in your deployment to reference the Kubernetes secret.

Do not enable synchronization if you do not want to store your secrets on your OKD cluster and in etcd. Enable this functionality only if you require it, such as when you want to use environment variables to refer to the secret.

If you enable synchronization, the secrets from the mounted volume are synchronized as Kubernetes secrets after you start a pod that mounts the secrets.

The synchronized Kubernetes secret is deleted when all pods that mounted the content are deleted.

Prerequisites
  • You have installed the Secrets Store CSI Driver Operator.

  • You have installed a secrets store provider.

  • You have created the secret provider class.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  1. Edit the SecretProviderClass resource by running the following command:

    $ oc edit secretproviderclass my-azure-provider (1)
    1 Replace my-azure-provider with the name of your secret provider class.
  2. Add the secretsObjects section with the configuration for the synchronized Kubernetes secrets:

    apiVersion: secrets-store.csi.x-k8s.io/v1
    kind: SecretProviderClass
    metadata:
      name: my-azure-provider
      namespace: my-namespace
    spec:
      provider: azure
      secretObjects:                                   (1)
        - secretName: tlssecret                        (2)
          type: kubernetes.io/tls                      (3)
          labels:
            environment: "test"
          data:
            - objectName: tlskey                       (4)
              key: tls.key                             (5)
            - objectName: tlscrt
              key: tls.crt
      parameters:
        usePodIdentity: "false"
        keyvaultName: "kvname"
        objects:  |
          array:
            - |
              objectName: tlskey
              objectType: secret
            - |
              objectName: tlscrt
              objectType: secret
        tenantId: "tid"
    1 Specify the configuration for synchronized Kubernetes secrets.
    2 Specify the name of the Kubernetes Secret object to create.
    3 Specify the type of Kubernetes Secret object to create. For example, Opaque or kubernetes.io/tls.
    4 Specify the object name or alias of the mounted content to synchronize.
    5 Specify the data field from the specified objectName to populate the Kubernetes secret with.
  3. Save the file to apply the changes.

Viewing the status of secrets in the pod volume mount

You can view detailed information, including the versions, of the secrets in the pod volume mount.

The Secrets Store CSI Driver Operator creates a SecretProviderClassPodStatus resource in the same namespace as the pod. You can review this resource to see detailed information, including versions, about the secrets in the pod volume mount.

Prerequisites
  • You have installed the Secrets Store CSI Driver Operator.

  • You have installed a secrets store provider.

  • You have created the secret provider class.

  • You have deployed a pod that mounts a volume from the Secrets Store CSI Driver Operator.

  • You have access to the cluster as a user with the cluster-admin role.

Procedure
  • View detailed information about the secrets in a pod volume mount by running the following command:

    $ oc get secretproviderclasspodstatus <secret_provider_class_pod_status_name> -o yaml (1)
    1 The name of the secret provider class pod status object is in the format of <pod_name>-<namespace>-<secret_provider_class_name>.
    Example output
    ...
    status:
      mounted: true
      objects:
      - id: secret/tlscrt
        version: f352293b97da4fa18d96a9528534cb33
      - id: secret/tlskey
        version: 02534bc3d5df481cb138f8b2a13951ef
      podName: busybox-<hash>
      secretProviderClassName: my-azure-provider
      targetPath: /var/lib/kubelet/pods/f0d49c1e-c87a-4beb-888f-37798456a3e7/volumes/kubernetes.io~csi/secrets-store-inline/mount

Uninstalling the Secrets Store CSI Driver Operator

Prerequisites
  • Access to the OKD web console.

  • Administrator access to the cluster.

Procedure

To uninstall the Secrets Store CSI Driver Operator:

  1. Stop all application pods that use the secrets-store.csi.k8s.io provider.

  2. Remove any third-party provider plug-in for your chosen secret store.

  3. Remove the Container Storage Interface (CSI) driver and associated manifests:

    1. Click AdministrationCustomResourceDefinitionsClusterCSIDriver.

    2. On the Instances tab, for secrets-store.csi.k8s.io, on the far left side, click the drop-down menu, and then click Delete ClusterCSIDriver.

    3. When prompted, click Delete.

  4. Verify that the CSI driver pods are no longer running.

  5. Uninstall the Secrets Store CSI Driver Operator:

    Before you can uninstall the Operator, you must remove the CSI driver first.

    1. Click OperatorsInstalled Operators.

    2. On the Installed Operators page, scroll or type "Secrets Store CSI" into the Search by name box to find the Operator, and then click it.

    3. On the upper, right of the Installed Operators > Operator details page, click ActionsUninstall Operator.

    4. When prompted on the Uninstall Operator window, click the Uninstall button to remove the Operator from the namespace. Any applications deployed by the Operator on the cluster need to be cleaned up manually.

      After uninstalling, the Secrets Store CSI Driver Operator is no longer listed in the Installed Operators section of the web console.