Hosted control planes release notes

Before this update, the SAN validation for custom certificates in hc.spec.configuration.apiServer.servingCerts.namedCertificates did not properly handle wildcard DNS patterns, such as *.example.com. As a consequence, the wildcard DNS patterns in custom certificates could conflict with internal Kubernetes API server certificate SANs without being detected, leading to certificate validation failures and potential deployment issues. This release provides enhanced DNS SAN conflict detection to include RFC-compliant wildcard support, implementing bidirectional conflict validation that properly handles wildcard patterns such as *.example.com matching sub.example.com. As a result, wildcard DNS patterns are now properly validated, preventing certificate conflicts and ensuring more reliable hosted cluster deployments with wildcard certificate support. (OCPBUGS-60381)

Before this update, the Azure cloud provider did not set the default ping target, HTTP:10256/healthz, for the Azure load balancer. Instead, services of the LoadBalancer type that ran on Azure had a ping target of TCP:30810. As a consequence, the health probes for cluster-wide services were non-functional, and during upgrades, they experienced downtime. With this release, the ClusterServiceLoadBalancerHealthProbeMode property of the cloud configuration is set to shared. As a result, load balancers in Azure have the correct health check ping target, HTTP:10256/healthz, which points to kube-proxy health endpoints that run on nodes. (OCPBUGS-58031)

Before this update, the HyperShift Operator failed to clear the user-ca-bundle config map after the removal of the additionalTrustBundle parameter from the HostedCluster resource. As a consequence, the user-ca-bundle config map was not updated, resulting in failure to generate ignition payloads. With this release, the HyperShift Operator actively removes the user-ca-bundle config map from the control plane namespace when it is removed from the HostedCluster resource. As a result, the user-ca-bundle config map is now correctly cleared, enabling the generation of ignition payloads. (OCPBUGS-57336)

Before this update, if you tried to create a hosted cluster on AWS when the Kubernetes API server service publishing strategy was LoadBalancer with PublicAndPrivate endpoint access, a private router admitted the OAuth route even though the External DNS Operator did not register a DNS record. As a consequence, the private router did not properly resolve the route URL and the OAuth server was inaccessible. The Console Cluster Operator also failed to start, and the hosted cluster installation failed. With this release, a private router admits the OAuth route only when the external DNS is defined. Otherwise, the router admits the route in the management cluster. As a result, the OAuth route is accessible, the Console Cluster Operator properly starts, and the hosted cluster installation succeeds. (OCPBUGS-56914)

Before this release, when an IDMS or ICSP in the management OpenShift cluster defined a source that pointed to registry.redhat.io or registry.redhat.io/redhat, and the mirror registry did not contain the required OLM catalog images, provisioning for the HostedCluster resource stalled due to unauthorized image pulls. As a consequence, the HostedCluster resource was not deployed, and it remained blocked, where it could not pull essential catalog images from the mirrored registry. With this release, if a required image cannot be pulled due to authorization errors, the provisioning now explicitly fails. The logic for registry override is improved to allow matches on the root of the registry, such as registry.redhat.io, for OLM CatalogSource image resolution. A fallback mechanism is also introduced to use the original ImageReference if the registry override does not yield a working image. As a result, the HostedCluster resource can be deployed successfully, even in scenarios where the mirror registry lacks the required OLM catalog images, as the system correctly falls back to pulling from the original source when appropriate. (OCPBUGS-56492)

Before this update, the AWS Cloud Provider did not set the default ping target, HTTP:10256/healthz, for the AWS load balancer. For services of the LoadBalancer type that run on AWS, the load balancer object created in AWS had a ping target of TCP:32518. As a consequence, the health probes for cluster-wide services were non-functional, and during upgrades, those services were down. With this release, the ClusterServiceLoadBalancerHealthProbeMode property of the cloud configuration is set to Shared. This cloud configuration is passed to the AWS Cloud Provider. As a result, the load balancers in AWS have the correct health check ping target, HTTP:10256/healthz, which points to the kube-proxy health endpoints that are running on nodes. (OCPBUGS-56011)

Before this update, when you disabled the image registry capability by using the --disable-cluster-capabilities option, hosted control planes still required you to configure a managed identity for the image registry. In this release, when the image registry is disabled, the image registry managed identity configuration is optional. (OCPBUGS-55892)

Before this update, the ImageDigestMirrorSet (IDMS) and ImageContentSourcePolicy (ICSP) resources from the management cluster were processed without considering that someone might specify only the root registry name as a mirror or source for image replacement. As a consequence, the IDMS and ICSP entries that used only the root registry name did not work as expected. In this release, the mirror replacement logic now correctly handles cases where only the root registry name is provided. As a result, the issue no longer occurs, and root registry mirror replacements are now supported. (OCPBUGS-54483)

Before this update, hosted control planes did not correctly persist registry metadata and release image provider caches in the HostedCluster resource. As a consequence, caches for release and image metadata reset on HostedCluster controller reconciliation. This release introduces a common registry provider which is used by the HostedCluster resource to fix cache loss. This reduces the number of image pulls and network traffic, thus improving overall performance. (OCPBUGS-53259)

Before this update, when you configured an OIDC provider for a HostedCluster resource with an OIDC client that did not specify a client secret, the system automatically generated a default secret name. As a consequence, you could not configure OIDC public clients, which are not supposed to use secrets. This release fixes the issue. If no client secret is provided, no default secret name is generated, enabling proper support for public clients. (OCPBUGS-58149)

Before this update, multiple mirror images caused a hosted control plane payload error due to failed image lookup. As a consequence, users could not create hosted clusters. With this release, the hosted control plane payload now supports multiple mirrors, avoiding errors when a primary mirror is unavailable. As a result, users can create hosted clusters. (OCPBUGS-54720)

Before this update, when a hosted cluster was upgraded to multiple versions over time, the version history in the HostedCluster resource sometimes exceeded 10 entries. However, the API had a strict validation limit of 10 items maximum for the version history field. As a consequence, users could not edit or update their HostedCluster resources when the version history exceeded 10 entries. Operations such as adding annotations (for example, for cluster size overrides) or performing maintenance tasks like resizing request serving nodes failed with a validation error: "status.version.history: Too many: 11: must have at most 10 items". This error prevented ROSA SREs from performing critical maintenance operations that might impact customer API access.

Before this update, following a CLI refactoring, the MarkPersistentFlagRequired function stopped working correctly. The --name and --pull-secret flags, which are critical for cluster creation, were marked as required, but the validation was not being enforced. As a consequence, users could run the hypershift create cluster command without providing the required --name or --pull-secret flags, and the CLI would not immediately alert them that these required flags were missing. This could lead to misconfigured deployments and confusing error messages later in the process.

Before this update, a variable shadowing bug in the GetSupportedOCPVersions() function caused the supportedVersions variable to be incorrectly assigned using := instead of =, creating a local variable that was immediately discarded rather than updating the intended outer scope variable. As a consequence, when users ran the hypershift version command with the HyperShift Operator deployed, the CLI would either display <unknown> for the Server Version or panic with a "nil pointer dereference" error, preventing users from verifying the deployed HyperShift Operator version.

Before this update, the HyperShift Operator validated the Kubernetes API Server subject alternative names (SANs) in all cases. As a consequence, users sometimes experienced invalid API Server SANs during public key infrastructure (PKI) reconciliation. With this release, the Kubernetes API Server SANs are validated only if PKI reconciliation is not disabled. (OCPBUGS-56457)

Before this update, the shared ingress controller did not handle the HostedCluster.Spec.KubeAPIServerDNSName field, so custom kube-apiserver DNS names were not added to the router configuration. As a consequence, traffic destined for the kube-apiserver on a hosted control plane that used a custom DNS name (via HostedCluster.Spec.KubeAPIServerDNSName) was not routed correctly, preventing the KubeAPIExternalName feature from working with platforms that use shared ingress.

If the annotation and the ManagedCluster resource name do not match, the multicluster engine for Kubernetes Operator console displays the cluster as Pending import. The cluster cannot be used by the multicluster engine Operator. The same issue happens when there is no annotation and the ManagedCluster name does not match the Infra-ID value of the HostedCluster resource.

When you use the multicluster engine for Kubernetes Operator console to add a new node pool to an existing hosted cluster, the same version of OKD might appear more than once in the list of options. You can select any instance in the list for the version that you want.

When a node pool is scaled down to 0 workers, the list of hosts in the console still shows nodes in a Ready state. You can verify the number of nodes in two ways:

When you create a hosted cluster in an environment that uses the dual-stack network, you might encounter pods stuck in the ContainerCreating state. This issue occurs because the openshift-service-ca-operator resource cannot generate the metrics-tls secret that the DNS pods need for DNS resolution. As a result, the pods cannot resolve the Kubernetes API server. To resolve this issue, configure the DNS server settings for a dual stack network.

If you created a hosted cluster in the same namespace as its managed cluster, detaching the managed hosted cluster deletes everything in the managed cluster namespace including the hosted cluster. The following situations can create a hosted cluster in the same namespace as its managed cluster:

When you use the console or API to specify an IPv6 address for the spec.services.servicePublishingStrategy.nodePort.address field of a hosted cluster, a full IPv6 address with 8 hextets is required. For example, instead of specifying 2620:52:0:1306::30, you need to specify 2620:52:0:1306:0:0:0:30.