OKD 4.9 uses Kubernetes 1.22, which removed a significant number of deprecated v1beta1 APIs.

OKD 4.8.14 introduced a requirement that an administrator must provide a manual acknowledgment before the cluster can be upgraded from OKD 4.8 to 4.9. This is to help prevent issues after upgrading to OKD 4.9, where APIs that have been removed are still in use by workloads, tools, or other components running on or interacting with the cluster. Administrators must evaluate their cluster for any APIs in use that will be removed and migrate the affected components to use the appropriate new API version. After this evaluation and migration is complete, the administrator can provide the acknowledgment.

Before you can upgrade your OKD 4.8 cluster to 4.9, you must provide the administrator acknowledgment.

Removed Kubernetes APIs

OKD 4.9 uses Kubernetes 1.22, which removed the following deprecated v1beta1 APIs. You must migrate manifests and API clients to use the v1 API version. For more information about migrating removed APIs, see the Kubernetes documentation.

Table 1. v1beta1 APIs removed from Kubernetes 1.22
Resource API Notable changes

APIService

apiregistration.k8s.io/v1beta1

No

CertificateSigningRequest

certificates.k8s.io/v1beta1

Yes

ClusterRole

rbac.authorization.k8s.io/v1beta1

No

ClusterRoleBinding

rbac.authorization.k8s.io/v1beta1

No

CSIDriver

storage.k8s.io/v1beta1

No

CSINode

storage.k8s.io/v1beta1

No

CustomResourceDefinition

apiextensions.k8s.io/v1beta1

Yes

Ingress

extensions/v1beta1

Yes

Ingress

networking.k8s.io/v1beta1

Yes

IngressClass

networking.k8s.io/v1beta1

No

Lease

coordination.k8s.io/v1beta1

No

LocalSubjectAccessReview

authorization.k8s.io/v1beta1

Yes

MutatingWebhookConfiguration

admissionregistration.k8s.io/v1beta1

Yes

PriorityClass

scheduling.k8s.io/v1beta1

No

Role

rbac.authorization.k8s.io/v1beta1

No

RoleBinding

rbac.authorization.k8s.io/v1beta1

No

SelfSubjectAccessReview

authorization.k8s.io/v1beta1

Yes

StorageClass

storage.k8s.io/v1beta1

No

SubjectAccessReview

authorization.k8s.io/v1beta1

Yes

TokenReview

authentication.k8s.io/v1beta1

No

ValidatingWebhookConfiguration

admissionregistration.k8s.io/v1beta1

Yes

VolumeAttachment

storage.k8s.io/v1beta1

No

Evaluating your cluster for removed APIs

There are several methods to help administrators identify where APIs that will be removed are in use. However, OKD cannot identify all instances, especially workloads that are idle or external tools that are used. It is the responsibility of the administrator to properly evaluate all workloads and other integrations for instances of removed APIs.

Reviewing alerts to identify uses of removed APIs

Two alerts fire when an API is in use that will be removed in the next release:

  • APIRemovedInNextReleaseInUse - for APIs that will be removed in the next OKD release.

  • APIRemovedInNextEUSReleaseInUse - for APIs that will be removed in the next OKD Extended Update Support (EUS) release.

If either of these alerts are firing in your cluster, review the alerts and take action to clear the alerts by migrating manifests and API clients to use the new API version. You can use the APIRequestCount API to get more information about which APIs are in use and which workloads are using removed APIs.

Using APIRequestCount to identify uses of removed APIs

You can use the APIRequestCount API to track API requests and review whether any of them are using one of the removed APIs.

Prerequisites
  • You must have access to the cluster as a user with the cluster-admin role.

Procedure
  • Run the following command and examine the REMOVEDINRELEASE column of the output to identify the removed APIs that are currently in use:

    $ oc get apirequestcounts
    Example output
    NAME                                        REMOVEDINRELEASE   REQUESTSINCURRENTHOUR   REQUESTSINLAST24H
    cloudcredentials.v1.operator.openshift.io                      32                      111
    ingresses.v1.networking.k8s.io                                 28                      110
    ingresses.v1beta1.extensions                1.22               16                      66
    ingresses.v1beta1.networking.k8s.io         1.22               0                       1
    installplans.v1alpha1.operators.coreos.com                     93                      167
    ...

    You can also use -o jsonpath to filter the results:

    $ oc get apirequestcounts -o jsonpath='{range .items[?(@.status.removedInRelease!="")]}{.status.removedInRelease}{"\t"}{.metadata.name}{"\n"}{end}'
    Example output
    1.22    certificatesigningrequests.v1beta1.certificates.k8s.io
    1.22    ingresses.v1beta1.extensions
    1.22    ingresses.v1beta1.networking.k8s.io

Using APIRequestCount to identify which workloads are using the removed APIs

You can examine the APIRequestCount resource for a given API version to help identify which workloads are using the API.

Prerequisites
  • You must have access to the cluster as a user with the cluster-admin role.

Procedure
  • Run the following command and examine the username and userAgent fields to help identify the workloads that are using the API:

    $ oc get apirequestcounts <resource>.<version>.<group> -o yaml

    For example:

    $ oc get apirequestcounts ingresses.v1beta1.networking.k8s.io -o yaml

    You can also use -o jsonpath to extract the username values from an APIRequestCount resource:

    $ oc get apirequestcounts ingresses.v1beta1.networking.k8s.io -o jsonpath='{range ..username}{$}{"\n"}{end}' | sort | uniq
    Example output
    user1
    user2
    app:serviceaccount:delta

Migrating instances of removed APIs

For information on how to migrate removed Kubernetes APIs, see the Deprecated API Migration Guide in the Kubernetes documentation.

Providing the administrator acknowledgment

After you have evaluated your cluster for any removed APIs and have migrated any removed APIs, you can acknowledge that your cluster is ready to upgrade from OKD 4.8 to 4.9.

Be aware that all responsibility falls on the administrator to ensure that all uses of removed APIs have been resolved and migrated as necessary before providing this administrator acknowledgment. OKD can assist with the evaluation, but cannot identify all possible uses of removed APIs, especially idle workloads or external tools.

Prerequisites
  • You must have access to the cluster as a user with the cluster-admin role.

Procedure
  • Run the following command to acknowledge that you have completed the evaluation and your cluster is ready to upgrade to OKD 4.9:

    $ oc -n openshift-config patch cm admin-acks --patch '{"data":{"ack-4.8-kube-1.22-api-removals-in-4.9":"true"}}' --type=merge