1. Documentation
×

Shared Resource CSI Driver Operator

The Shared Resource CSI Driver feature is now generally available in Shipwright 1.1. This feature is now deprecated in OKD. To use this feature, ensure you are using Shipwright 1.1 or a more recent version.

As a cluster administrator, you can use the Shared Resource CSI Driver in OKD to provision inline ephemeral volumes that contain the contents of Secret or ConfigMap objects. This way, pods and other Kubernetes types that expose volume mounts, and OKD Builds can securely use the contents of those objects across potentially any namespace in the cluster. To accomplish this, there are currently two types of shared resources: a SharedSecret custom resource for Secret objects, and a SharedConfigMap custom resource for ConfigMap objects.

To enable the Shared Resource CSI Driver, you must enable features using feature gates.

About CSI

Storage vendors have traditionally provided storage drivers as part of Kubernetes. With the implementation of the Container Storage Interface (CSI), third-party providers can instead deliver storage plugins using a standard interface without ever having to change the core Kubernetes code.

CSI Operators give OKD users storage options, such as volume snapshots, that are not possible with in-tree volume plugins.

Sharing secrets across namespaces

To share a secret across namespaces in a cluster, you create a SharedSecret custom resource (CR) instance for the Secret object that you want to share.

Prerequisites

You must have permission to perform the following actions:

  • Create instances of the sharedsecrets.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

  • Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

  • Manage roles and role bindings to control whether the service account specified by a pod can mount a Container Storage Interface (CSI) volume that references the SharedSecret CR instance you want to use.

  • Access the namespaces that contain the Secrets you want to share.

Procedure

  • Create a SharedSecret CR instance for the Secret object you want to share across namespaces in the cluster:

    $ oc apply -f - <<EOF
apiVersion: sharedresource.openshift.io/v1alpha1
kind: SharedSecret
metadata:
  name: my-share
spec:
  secretRef:
    name: <name of secret>
    namespace: <namespace of secret>
EOF

Using a SharedSecret instance in a pod

To access a SharedSecret custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedSecret CR instance.

Prerequisites

  • You have created a SharedSecret CR instance for the secret you want to share across namespaces in the cluster.

  • You must have permission to perform the following actions

    • Discover which SharedSecret CR instances are available by entering the oc get sharedsecrets command and getting a non-empty list back.

    • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

    • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See "Understanding and managing pod security admission" for details.

If neither of the last two prerequisites in this list are met, create, or ask someone to create, the necessary role-based access control (RBAC) so that you can discover SharedSecret CR instances and enable service accounts to use SharedSecret CR instances.
Procedure

  1. Grant a given service account RBAC permissions to use the SharedSecret CR instance in its pod by using oc apply with YAML content:

    Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming SharedSecret CR instances.

    		 
    $ oc apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: shared-resource-my-share
  namespace: my-namespace
rules:
  - apiGroups:
      - sharedresource.openshift.io
    resources:
      - sharedsecrets
    resourceNames:
      - my-share
    verbs:
      - use
EOF

  2. Create the RoleBinding associated with the role by using the oc command:

    $ oc create rolebinding shared-resource-my-share --role=shared-resource-my-share --serviceaccount=my-namespace:builder

  3. Access the SharedSecret CR instance from a pod:

    $ oc apply -f - <<EOF
kind: Pod
apiVersion: v1
metadata:
  name: my-app
  namespace: my-namespace
spec:
  serviceAccountName: default

# containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume

    volumes:
    - name: my-csi-volume
      csi:
        readOnly: true
        driver: csi.sharedresource.openshift.io
        volumeAttributes:
          sharedSecret: my-share

EOF

Sharing a config map across namespaces

To share a config map across namespaces in a cluster, you create a SharedConfigMap custom resource (CR) instance for that config map.

Prerequisites

You must have permission to perform the following actions:

  • Create instances of the sharedconfigmaps.sharedresource.openshift.io custom resource definition (CRD) at a cluster-scoped level.

  • Manage roles and role bindings across the namespaces in the cluster to control which users can get, list, and watch those instances.

  • Manage roles and role bindings across the namespaces in the cluster to control which service accounts in pods that mount your Container Storage Interface (CSI) volume can use those instances.

  • Access the namespaces that contain the Secrets you want to share.

Procedure

  1. Create a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster:

    $ oc apply -f - <<EOF
apiVersion: sharedresource.openshift.io/v1alpha1
kind: SharedConfigMap
metadata:
  name: my-share
spec:
  configMapRef:
    name: <name of configmap>
    namespace: <namespace of configmap>
EOF

Using a SharedConfigMap instance in a pod

Next steps

To access a SharedConfigMap custom resource (CR) instance from a pod, you grant a given service account RBAC permissions to use that SharedConfigMap CR instance.

Prerequisites

  • You have created a SharedConfigMap CR instance for the config map that you want to share across namespaces in the cluster.

  • You must have permission to perform the following actions:

    • Discover which SharedConfigMap CR instances are available by entering the oc get sharedconfigmaps command and getting a non-empty list back.

    • Determine if the service account your pod specifies is allowed to use the given SharedSecret CR instance. That is, you can run oc adm policy who-can use <identifier of specific SharedSecret> to see if the service account in your namespace is listed.

    • Determine if the service account your pod specifies is allowed to use csi volumes, or if you, as the requesting user who created the pod directly, are allowed to use csi volumes. See "Understanding and managing pod security admission" for details.

If neither of the last two prerequisites in this list are met, create, or ask someone to create, the necessary role-based access control (RBAC) so that you can discover SharedConfigMap CR instances and enable service accounts to use SharedConfigMap CR instances.
Procedure

  1. Grant a given service account RBAC permissions to use the SharedConfigMap CR instance in its pod by using oc apply with YAML content.

    Currently, kubectl and oc have hard-coded special case logic restricting the use verb to roles centered around pod security. Therefore, you cannot use oc create role …​ to create the role needed for consuming a SharedConfigMap CR instance.

    		 
    $ oc apply -f - <<EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: shared-resource-my-share
  namespace: my-namespace
rules:
  - apiGroups:
      - sharedresource.openshift.io
    resources:
      - sharedconfigmaps
    resourceNames:
      - my-share
    verbs:
      - use
EOF

  2. Create the RoleBinding associated with the role by using the oc command:

    oc create rolebinding shared-resource-my-share --role=shared-resource-my-share --serviceaccount=my-namespace:builder

  3. Access the SharedConfigMap CR instance from a pod:

    $ oc apply -f - <<EOF
kind: Pod
apiVersion: v1
metadata:
  name: my-app
  namespace: my-namespace
spec:
  serviceAccountName: default

# containers omitted …. Follow standard use of ‘volumeMounts’ for referencing your shared resource volume

    volumes:
    - name: my-csi-volume
      csi:
        readOnly: true
        driver: csi.sharedresource.openshift.io
        volumeAttributes:
          sharedConfigMap: my-share

EOF

Additional support limitations for the Shared Resource CSI Driver

The Shared Resource CSI Driver has the following noteworthy limitations:

  • The driver is subject to the limitations of Container Storage Interface (CSI) inline ephemeral volumes.

  • The value of the readOnly field must be true. On Pod creation, a validating admission webhook rejects the pod creation if readOnly is false. If for some reason the validating admission webhook cannot be contacted, on volume provisioning during pod startup, the driver returns an error to the kubelet. Requiring readOnly is true is in keeping with proposed best practices for the upstream Kubernetes CSI Driver to apply SELinux labels to associated volumes.

  • The driver ignores the FSType field because it only supports tmpfs volumes.

  • The driver ignores the NodePublishSecretRef field. Instead, it uses SubjectAccessReviews with the use verb to evaluate whether a pod can obtain a volume that contains SharedSecret or SharedConfigMap custom resource (CR) instances.

  • You cannot create SharedSecret or SharedConfigMap custom resource (CR) instances whose names start with openshift.

Additional details about VolumeAttributes on shared resource pod volumes

The Shared Resource CSI Driver feature is now generally available in Shipwright 1.1. This feature is now deprecated in OKD. To use this feature, ensure you are using Shipwright 1.1 or a more recent version.

The following attributes affect shared resource pod volumes in various ways:

  • The refreshResource attribute in the volumeAttributes properties.

  • The refreshResources attribute in the Shared Resource CSI Driver configuration.

  • The sharedSecret and sharedConfigMap attributes in the volumeAttributes properties.

The refreshResource attribute

The Shared Resource CSI Driver honors the refreshResource attribute in volumeAttributes properties of the volume. This attribute controls whether updates to the contents of the underlying Secret or ConfigMap object are copied to the volume after the volume is initially provisioned as part of pod startup. The default value of refreshResource is true, which means that the contents are updated.

If the Shared Resource CSI Driver configuration has disabled the refreshing of both the shared SharedSecret and SharedConfigMap custom resource (CR) instances, then the refreshResource attribute in the volumeAttribute properties has no effect. The intent of this attribute is to disable refresh for specific volume mounts when refresh is generally allowed.

The refreshResources attribute

You can use a global switch to enable or disable refreshing of shared resources. This switch is the refreshResources attribute in the csi-driver-shared-resource-config config map for the Shared Resource CSI Driver, which you can find in the openshift-cluster-csi-drivers namespace. If you set this refreshResources attribute to false, none of the Secret or ConfigMap object-related content stored in the volume is updated after the initial provisioning of the volume.

Using this Shared Resource CSI Driver configuration to disable refreshing affects all the cluster’s volume mounts that use the Shared Resource CSI Driver, regardless of the refreshResource attribute in the volumeAttributes properties of any of those volumes.

Validation of volumeAttributes before provisioning a shared resource volume for a pod

In the volumeAttributes of a single volume, you must set either a sharedSecret or a sharedConfigMap attribute to the value of a SharedSecret or a SharedConfigMap CS instance. Otherwise, when the volume is provisioned during pod startup, a validation checks the volumeAttributes of that volume and returns an error to the kubelet under the following conditions:

  • Both sharedSecret and sharedConfigMap attributes have specified values.

  • Neither sharedSecret nor sharedConfigMap attributes have specified values.

  • The value of the sharedSecret or sharedConfigMap attribute does not correspond to the name of a SharedSecret or SharedConfigMap CR instance on the cluster.

Integration between shared resources, Insights Operator, and OKD Builds

Integration between shared resources, Insights Operator, and OKD Builds makes using Red Hat subscriptions (RHEL entitlements) easier in OKD Builds.

Previously, in OKD 4.9.x and earlier, you manually imported your credentials and copied them to each project or namespace where you were running builds.

Now, in OKD 4.10 and later, OKD Builds can use Red Hat subscriptions (RHEL entitlements) by referencing shared resources and the simple content access feature provided by Insights Operator:

  • The simple content access feature imports your subscription credentials to a well-known Secret object. See the links in the following "Additional resources" section.

  • The cluster administrator creates a SharedSecret custom resource (CR) instance around that Secret object and grants permission to particular projects or namespaces. In particular, the cluster administrator gives the builder service account permission to use that SharedSecret CR instance.

  • Builds that run within those projects or namespaces can mount a CSI Volume that references the SharedSecret CR instance and its entitled RHEL content.

Additional resources