In OKD version 4.12, you can install a customized OKD cluster on oVirt and other infrastructure that you provide. The OKD documentation uses the term user-provisioned infrastructure to refer to this infrastructure type.
The following diagram shows an example of a potential OKD cluster running on a oVirt cluster.
The oVirt hosts run virtual machines that contain both control plane and compute pods. One of the hosts also runs a Engine virtual machine and a bootstrap virtual machine that contains a temporary control plane pod.]
The following items are required to install an OKD cluster on a oVirt environment.
You reviewed details about the OKD installation and update processes.
You read the documentation on selecting a cluster installation method and preparing it for users.
You have a supported combination of versions in the Support Matrix for OKD on oVirt.
To install and run an OKD version 4.12 cluster, the oVirt environment must meet the following requirements.
Not meeting these requirements can cause the installation or process to fail. Additionally, not meeting these requirements can cause the OKD cluster to fail days or weeks after installation.
The following requirements for CPU, memory, and storage resources are based on default values multiplied by the default number of virtual machines the installation program creates. These resources must be available in addition to what the oVirt environment uses for non-OKD operations.
By default, the installation program creates seven virtual machines during the installation process. First, it creates a bootstrap virtual machine to provide temporary services and a control plane while it creates the rest of the OKD cluster. When the installation program finishes creating the cluster, deleting the bootstrap machine frees up its resources.
If you increase the number of virtual machines in the oVirt environment, you must increase the resources accordingly.
The oVirt version is 4.4.
The oVirt environment has one data center whose state is Up.
The oVirt data center contains an oVirt cluster.
The oVirt cluster has the following resources exclusively for the OKD cluster:
Minimum 28 vCPUs: four for each of the seven virtual machines created during installation.
112 GiB RAM or more, including:
16 GiB or more for the bootstrap machine, which provides the temporary control plane.
16 GiB or more for each of the three control plane machines which provide the control plane.
16 GiB or more for each of the three compute machines, which run the application workloads.
The oVirt storage domain must meet these etcd backend performance requirements.
In production environments, each virtual machine must have 120 GiB or more. Therefore, the storage domain must provide 840 GiB or more for the default OKD cluster. In resource-constrained or non-production environments, each virtual machine must have 32 GiB or more, so the storage domain must have 230 GiB or more for the default OKD cluster.
To download images from the Red Hat Ecosystem Catalog during installation and update procedures, the oVirt cluster must have access to an internet connection. The Telemetry service also needs an internet connection to simplify the subscription and entitlement process.
The oVirt cluster must have a virtual network with access to the REST API on the oVirt Engine. Ensure that DHCP is enabled on this network, because the VMs that the installer creates obtain their IP address by using DHCP.
A user account and group with the following least privileges for installing and managing an OKD cluster on the target oVirt cluster:
ClusterAdmin on the target cluster
Apply the principle of least privilege: Avoid using an administrator account with
Verify that the oVirt environment meets the requirements to install and run an OKD cluster. Not meeting these requirements can cause failures.
These requirements are based on the default resources the installation program uses to create control plane and compute machines. These resources include vCPUs, memory, and storage. If you change these resources or increase the number of OKD machines, adjust these requirements accordingly.
Check that the oVirt version supports installation of OKD version 4.12.
In the oVirt Administration Portal, click the ? help icon in the upper-right corner and select About.
In the window that opens, make a note of the oVirt Software Version.
Confirm that the oVirt version is 4.4. For more information about supported version combinations, see Support Matrix for OKD on oVirt.
Inspect the data center, cluster, and storage.
In the oVirt Administration Portal, click Compute → Data Centers.
Confirm that the data center where you plan to install OKD is accessible.
Click the name of that data center.
In the data center details, on the Storage tab, confirm the storage domain where you plan to install OKD is Active.
Record the Domain Name for use later on.
Confirm Free Space has at least 230 GiB.
Confirm that the storage domain meets these etcd backend performance requirements, which you can measure by using the fio performance benchmarking tool.
In the data center details, click the Clusters tab.
Find the oVirt cluster where you plan to install OKD. Record the cluster name for use later on.
Inspect the oVirt host resources.
In the oVirt Administration Portal, click Compute > Clusters.
Click the cluster where you plan to install OKD.
In the cluster details, click the Hosts tab.
Inspect the hosts and confirm they have a combined total of at least 28 Logical CPU Cores available exclusively for the OKD cluster.
Record the number of available Logical CPU Cores for use later on.
Confirm that these CPU cores are distributed so that each of the seven virtual machines created during installation can have four cores.
Confirm that, all together, the hosts have 112 GiB of Max free Memory for scheduling new virtual machines distributed to meet the requirements for each of the following OKD machines:
16 GiB required for the bootstrap machine
16 GiB required for each of the three control plane machines
16 GiB for each of the three compute machines
Record the amount of Max free Memory for scheduling new virtual machines for use later on.
Verify that the virtual network for installing OKD has access to the oVirt Engine’s REST API. From a virtual machine on this network, use curl to reach the oVirt Engine’s REST API:
$ curl -k -u <username>@<profile>:<password> \ (1) https://<engine-fqdn>/ovirt-engine/api (2)
$ curl -k -u admin@internal:pw123 \ https://ovirtlab.example.com/ovirt-engine/api
All the Fedora CoreOS (FCOS) machines require networking to be configured in
initramfs during boot
to fetch their Ignition config files.
During the initial boot, the machines require an IP address configuration that is set either through a DHCP server or statically by providing the required boot options. After a network connection is established, the machines download their Ignition config files from an HTTP or HTTPS server. The Ignition config files are then used to set the exact state of each machine. The Machine Config Operator completes more changes to the machines, such as the application of new certificates or keys, after installation.
It is recommended to use a DHCP server for long-term management of the cluster machines. Ensure that the DHCP server is configured to provide persistent IP addresses, DNS server information, and hostnames to the cluster machines.
If a DHCP service is not available for your user-provisioned infrastructure, you can instead provide the IP networking configuration and the address of the DNS server to the nodes at FCOS install time. These can be passed as boot arguments if you are installing from an ISO image. See the Installing FCOS and starting the OKD bootstrap process section for more information about static IP provisioning and advanced networking options.
The Kubernetes API server must be able to resolve the node names of the cluster machines. If the API servers and worker nodes are in different zones, you can configure a default DNS search zone to allow the API server to resolve the node names. Another supported approach is to always refer to hosts by their fully-qualified domain names in both the node objects and all DNS requests.
Configure your firewall so your cluster has access to required sites.
Configure one or preferably two layer-4 load balancers:
Provide load balancing for ports
22623 on the control plane and bootstrap machines. Port
6443 provides access to the Kubernetes API server and must be reachable both internally and externally. Port
22623 must be accessible to nodes within the cluster.
Provide load balancing for port
80 for machines that run the Ingress router, which are usually compute nodes in the default configuration. Both ports must be accessible from within and outside the cluster.
Configure infrastructure-provided DNS to allow the correct resolution of the main components and services. If you use only one load balancer, these DNS records can point to the same IP address.
Create DNS records for
api.<cluster_name>.<base_domain> (internal and external resolution) and
api-int.<cluster_name>.<base_domain> (internal resolution) that point to the load balancer for the control plane machines.
Create a DNS record for
*.apps.<cluster_name>.<base_domain> that points to the load balancer for the Ingress router. For example, ports
80 of the compute machines.
On Fedora CoreOS (FCOS) machines, the hostname is set through NetworkManager. By default, the machines obtain their hostname through DHCP. If the hostname is not provided by DHCP, set statically through kernel arguments, or another method, it is obtained through a reverse DNS lookup. Reverse DNS lookup occurs after the network has been initialized on a node and can take time to resolve. Other system services can start prior to this and detect the hostname as
localhost or similar. You can avoid this by using DHCP to provide the hostname for each cluster node.
Additionally, setting the hostnames through DHCP can bypass any manual DNS record name configuration errors in environments that have a DNS split-horizon implementation.
You must configure the network connectivity between machines to allow OKD cluster components to communicate. Each machine must be able to resolve the hostnames of all other machines in the cluster.
This section provides details about the ports that are required.
In connected OKD environments, all nodes are required to have internet access to pull images for platform containers and provide telemetry data to Red Hat.
Network reachability tests
Host level services, including the node exporter on ports
The default ports that Kubernetes reserves
Host level services, including the node exporter on ports