Extending the Kubernetes API with Custom Resources

Kubernetes custom resource definitions
Creating a custom resource definition
Creating cluster roles for the custom resource definition
Creating custom objects from a CRD
Managing custom objects

Kubernetes custom resource definitions

In the Kubernetes API a resource is an endpoint that stores a collection of API objects of a certain kind. For example, the built-in pods resource contains a collection of Pod objects.

A custom resource is an object that extends the Kubernetes API or allows you to introduce your own API into a project or a cluster.

A custom resource definition (CRD) file defines your own object kinds and lets the API Server handle the entire lifecycle. Deploying a CRD into the cluster causes the Kubernetes API server to begin serving the specified custom resource.

When you create a new custom resource definition (CRD), the Kubernetes API Server reacts by creating a new RESTful resource path, that can be accessed by an entire cluster or a single project (namespace). As with existing built-in objects, deleting a project deletes all custom objects in that project.

If you want to grant access to the CRD to users, use cluster role aggregation to grant access to users with the admin, edit, or view default cluster roles. Cluster role aggregation allows the insertion of custom policy rules into these cluster roles. This behavior integrates the new resource into the cluster’s RBAC policy as if it was a built-in resource.

While only cluster admins can create CRDs, you can create an object from a CRD if you have read and write permission to it.

Creating a custom resource definition

To create custom objects, you must first create a custom resource definition (CRD).

Only cluster admins can create CRDs.

Procedure

To create a CRD:

Create a YAML file that contains the fields in the following example:

Example YAML file for a Custom Resource Definition

apiVersion: apiextensions.k8s.io/v1beta1 (1)
kind: CustomResourceDefinition
metadata:
  name: crontabs.stable.example.com (2)
spec:
  group: stable.example.com (3)
  version: v1 (4)
  scope: Namespaced (5)
  names:
    plural: crontabs (6)
    singular: crontab (7)
    kind: CronTab (8)
    shortNames:
    - ct (9)

1	Use the `apiextensions.k8s.io/v1beta1` API.
2	Specify a name for the definition. This must be in the <plural-name><group> format using the values from the `group` and `plural` fields.
3	Specify a group name for the API. An API group is a collection of objects that are logically related. For example, all batch objects like `Job` or `ScheduledJob` could be in the batch API Group (such as batch.api.example.com). A good practice is to use a fully-qualified-domain name of your organization.
4	Specify a version name to be used in the URL. Each API Group can exist in multiple versions. For example: `v1alpha`, `v1beta`, `v1`.
5	Specify whether the custom objects are available to a project (`Namespaced`) or all projects in the cluster (`Cluster`).
6	Specify the plural name to use in the URL. The `plural` field is the same as a resource in an API URL.
7	Specify a singular name to use as an alias on the CLI and for display.
8	Specify the kind of objects that can be created. The type can be in CamelCase.
9	Specify a shorter string to match your resource on the CLI.

By default, a custom resource definition is cluster-scoped and available to all projects.

Create the object:
```
oc create -f <file-name>.yaml
```
A new RESTful API endpoint is created at:
```
/apis/<spec:group>/<spec:version>/<scope>/*/<names-plural>/...
```
For example, using the example file, the following endpoint is created:
```
/apis/stable.example.com/v1/namespaces/*/crontabs/...
```
You can use this endpoint URL to create and manage custom objects. The kind of object is based on the spec.kind field of the Custom Resource Definition object you created.

Creating cluster roles for the custom resource definition

After you create a cluster-scoped custom resource definition (CRD), you can grant permissions to it. If you use the admin, edit, and view default cluster roles, take advantage of cluster role aggregation for their rules.

You must explicitly assign permissions to each of these roles. The roles with more permissions do not inherit rules from roles with fewer permissions. If you assign a rule to a role, you must also assign that verb to roles that have more permissions. For example, if you grant the "get crontabs" permission to the view role, you must also grant it to the edit and admin roles. The admin or edit role is usually assigned to the user that created a project through the project template.

Prerequisites

Create a CRD.

Procedure

Create a cluster role definition file for the CRD. The cluster role definition is a YAML file that contains the rules that apply to each cluster role. The OKD controller adds the rules that you specify to the default cluster roles.

Example YAML file for a cluster role definition

apiVersion: rbac.authorization.k8s.io/v1 (1)
kind: ClusterRole
items:
  - metadata:
      name: aggregate-cron-tabs-admin-edit (2)
      labels:
        rbac.authorization.k8s.io/aggregate-to-admin: "true" (3)
        rbac.authorization.k8s.io/aggregate-to-edit: "true" (4)
    rules:
      - apiGroups: ["stable.example.com"] (5)
        resources: ["crontabs"] (6)
        verbs: ["get", "list", "watch", "create",
                "update", "patch", "delete", "deletecollection"] (7)
  - metadata:
      name: aggregate-cron-tabs-view (2)
      labels:
        # Add these permissions to the "view" default role.
        rbac.authorization.k8s.io/aggregate-to-view: "true" (8)
        rbac.authorization.k8s.io/aggregate-to-cluster-reader: "true" (9)
    rules:
      - apiGroups: ["stable.example.com"] (5)
        resources: ["crontabs"] (6)
        verbs: ["get", "list", "watch"] (7)

1	Use the `apiextensions.k8s.io/v1beta1` API.
2	Specify a name for the definition.
3	Specify this label to grant permissions to the admin default role.
4	Specify this label to grant permissions to the edit default role.
5	Specify the group name of the CRD.
6	Specify the plural name of the CRD that these rules apply to.
7	Specify the verbs that represent the permissions that are granted to the role. For example, apply read and write permissions to the admin and edit roles and only read permission to the view role.
8	Specify this label to grant permissions to the view default role.
9	Specify this label to grant permissions to the cluster-reader default role.

Create the cluster role:
```
oc create -f <file-name>.yaml
```

Creating custom objects from a CRD

After you create the custom resource definition (CRD) object, you can create custom objects that use its specification.

Custom objects can contain custom fields that contain arbitrary JSON code.

Prerequisites

Create a CRD.

Procedure

Create a YAML definition for the custom object. In the following example definition, the cronSpec and image custom fields are set in a custom object of kind CronTab. The kind comes from the spec.kind field of the custom resource definition object.

Example YAML file for a custom object

apiVersion: "stable.example.com/v1" (1)
kind: CronTab (2)
metadata:
  name: my-new-cron-object (3)
  finalizers: (4)
  - finalizer.stable.example.com
spec: (5)
  cronSpec: "* * * * /5"
  image: my-awesome-cron-image

1	Specify the group name and API version (name/version) from the custom resource definition.
2	Specify the type in the custom resource definition.
3	Specify a name for the object.
4	Specify the finalizers for the object, if any. Finalizers allow controllers to implement conditions that must be completed before the object can be deleted.
5	Specify conditions specific to the type of object.

After you create the object file, create the object:
```
oc create -f <file-name>.yaml
```

Managing custom objects

After you create objects, you can manage your custom resources.

Prerequisites

Create a custom resource definition (CRD).
Create an object from a CRD.

Procedure

To get information on a specific kind of custom resource, enter:
```
oc get <kind>
```
For example:
```
oc get crontab

NAME                 KIND
my-new-cron-object   CronTab.v1.stable.example.com
```
Note that resource names are not case-sensitive, and you can use either the singular or plural forms defined in the CRD, as well as any short name. For example:
```
oc get crontabs
oc get crontab
oc get ct
```

You can also view the raw YAML data for a custom resource:

oc get <kind> -o yaml

oc get ct -o yaml

apiVersion: v1
items:
- apiVersion: stable.example.com/v1
  kind: CronTab
  metadata:
    clusterName: ""
    creationTimestamp: 2017-05-31T12:56:35Z
    deletionGracePeriodSeconds: null
    deletionTimestamp: null
    name: my-new-cron-object
    namespace: default
    resourceVersion: "285"
    selfLink: /apis/stable.example.com/v1/namespaces/default/crontabs/my-new-cron-object
    uid: 9423255b-4600-11e7-af6a-28d2447dc82b
  spec:
    cronSpec: '* * * * /5' (1)
    image: my-awesome-cron-image (1)

1	Custom data from the YAML that you used to create the object displays.

Kubernetes custom resource definitions

Creating a custom resource definition

Procedure

Creating cluster roles for the custom resource definition

Prerequisites

Procedure

Creating custom objects from a CRD

Prerequisites

Procedure

Managing custom objects

Prerequisites

Procedure