Overview

You can run multiple, custom schedulers alongside the default scheduler and configure which scheduler to use for each pods.

To schedule a given pod using a specific scheduler, specify the name of the scheduler in that pod specification.

Information on how to create the scheduler binary is outside the scope of this document. For an example, see Configure Multiple Schedulers in the Kubernetes documentation.

Package the Scheduler

The general process for including a custom scheduler in your cluster involves creating an image and including that image in a deployment.

  1. Package your scheduler binary into a container image.

  2. Create a container image containing the scheduler binary.

    For example:

    FROM <source-image>
    ADD <path-to-binary> /usr/local/bin/kube-scheduler
  3. Save the file as Dockerfile, build the image, and push it to a registry.

    For example:

    docker build -t <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
    docker push <dest_env_registry_ip>:<port>/<namespace>/<image name>:<tag>
  4. In OKD, create a deployment for the custom scheduler.

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: custom-scheduler
      namespace: kube-system
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: custom-scheduler
    subjects:
    - kind: ServiceAccount
      name: custom-scheduler
      namespace: kube-system
    roleRef:
      kind: ClusterRole
      name: system:kube-scheduler
      apiGroup: rbac.authorization.k8s.io
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: custom-scheduler
      namespace: kube-system
      labels:
        app: custom-scheduler
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: custom-scheduler
      template:
        metadata:
          labels:
            app: custom-scheduler
        spec:
          serviceAccount: custom-scheduler
          containers:
            - name: custom-scheduler
              image: "<namespace>/<image name>:<tag>" (1)
              imagePullPolicy: Always
    1 Specify the container image you created for the custom scheduler.

Deploying Pods using a Custom Scheduler

After your custom scheduler is deployed in your cluster, you can configure pods to use that scheduler instead of the default scheduler.

  1. Create or edit a pod configuration and specify the name of the scheduler with the schedulerName parameter. The name must be unique.

    Sample pod specification with scheduler
    apiVersion: v1
    kind: Pod
    metadata:
      name: custom-scheduler-example
      labels:
        name: custom-scheduler-example
    spec:
      schedulerName: custom-scheduler (1)
      containers:
      - name: pod-with-second-annotation-container
        image: docker.io/ocpqe/hello-pod
    1 The name of the scheduler to use. When no scheduler name is supplied, the pod is automatically scheduled using the default scheduler.
  2. Run the following command to create the pod:

    $ oc create -f <file-name>.yaml

    For example:

    $ oc create -f custom-scheduler-example.yaml
  3. Run the following command to check that the pod was created:

    $ oc get pod <file-name>

    For example:

    $ oc get pod custom-scheduler-example
    
    NAME                       READY     STATUS    RESTARTS   AGE
    custom-scheduler-example   1/1       Running   0          4m
  4. Run the following command to check that the custom scheduler scheduled the pod:

    $ oc describe pod <pod-name>

    For example:

    $ oc describe pod custom-scheduler-example

    The name of the scheduler is listed, as shown in the following truncated output:

    ...
    
    Events:
      FirstSeen  LastSeen  Count  From                SubObjectPath  Type       Reason Message
      ---------  --------  -----  ----                -------------  --------   ------ -------
      1m         1m        1      custom-scheduler    Normal         Scheduled  Successfully assigned custom-scheduler to <$node1>
    
    ...