1. Documentation
×

Expanding Persistent Volumes

Enabling Expansion of Persistent Volume Claims

Volume expansion is a Technology Preview feature, and hence, is not enabled by default in your OKD 3.9 cluster. There may be other reasons that OKD administrators wish to enable this feature for certain use cases.

For more information on Red Hat Technology Preview features support scope, see https://access.redhat.com/support/offerings/techpreview/.

To allow expansion of persistent volume claims (PVC) by OKD users, OKD administrators must create or update a StorageClass with allowVolumeExpansion set to true. Only PVCs created from that class are allowed to expand.

Apart from that, OKD administrators must enable the ExpandPersistentVolumes feature flag and turn on the PersistentVolumeClaimResize admission controller. Refer to Admission Controllers for more information on the PersistentVolumeClaimResize admission controller.

To enable the feature gate, set ExpandPersistentVolumes to true across the system:

  1. Configure node-config.yaml on all nodes in the cluster:

    # cat /etc/origin/node/node-config.yaml
...
kubeletArguments:
...
  feature-gates:
  - ExpandPersistentVolumes=true
# systemctl restart atomic-openshift-node

  2. Enable the ExpandPersistentVolumes feature gate on the master API and controller manager:

    # cat /etc/origin/master/master-config.yaml
...
kubernetesMasterConfig:
  apiServerArguments:
  ...
    feature-gates:
    - ExpandPersistentVolumes=true
  controllerArguments:
    ...
    feature-gates:
    - ExpandPersistentVolumes=true

# systemctl restart atomic-openshift-master-api

Expanding GlusterFS-Based Persistent Volume Claims

Once the OKD administrator has created a StorageClass with allowVolumeExpansion set to true, you can create a PVC from that class, and afterwards, whenever needed, you can edit the PVC and request a new size.

For example:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: gluster-mysql
spec:
  storageClass: "storageClassWithFlagSet"
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi (1)
1 You can request an expanded volume by updating spec.resources.requests.

Expanding Persistent Volume Claims with a File System

Expanding PVCs based on volume types that need file system resizing (such as GCE PD, EBS, and Cinder) is a two-step process. This process usually involves expanding volume objects in the CloudProvider, and then expanding the file system on the actual node.

Expanding the file system on the node only happens when a new pod is started with the volume.

The following process assumes that the PVC was previously created from a StorageClass with allowVolumeExpansion set to true:

  1. Edit the PVC and request a new size by editing spec.resources.requests. Once the CloudProvider object has finished resizing, the PVC is set to FileSystemResizePending.

  2. Type the following command to check the condition:

  oc describe pvc <pvc_name>

When the CloudProvider object has finished resizing, the persistent volume (PV) object reflects the newly requested size in PersistentVolume.Spec.Capacity. At this point, you can create or re-create a new pod from the PVC to finish the file system resizing. Once the pod is running, the newly requested size is available and FileSystemResizePending condition is removed from the PVC.

Recovering from Failure when Expanding Volumes

If expanding underlying storage fails either on master or node, the OKD administrator can manually recover the PVC state and cancel the resize requests that are continuously retried by the controller without administrator intervention.

Currently, this can be done manually by completing the following steps:

  1. Mark the PV that is bound to the claim (PVC) with the Retain reclaim policy. This can be done by editing the PV and changing persistentVolumeReclaimPolicy to Retain.

  2. Delete the PVC (it will be recreated later).

  3. To ensure that the newly created PVC can bind to the PV marked Retain, manually edit the PV and delete the claimRef entry from the PV specs. This marks the PV as Available. For more information about prebinding PVCs, see volume and claim prebinding.

  4. Re-create the PVC in a smaller size or a size that can be allocated by the underlying storage provider. Also, set the volumeName field of the PVC to the name of the PV. This binds the PVC to the provisioned PV only.

  5. Restore the reclaim policy on the PV.