apiVersion: v1
kind: Namespace
metadata:
name: openshift-windows-machine-config-operator (1)
labels:
openshift.io/cluster-monitoring: "true" (2)
Before adding Windows workloads to your cluster, you must install the Windows Machine Config Operator (WMCO), which is available in the OKD OperatorHub. The WMCO orchestrates the process of deploying and managing Windows workloads on a cluster.
Dual NIC is not supported on WMCO-managed Windows instances. |
You have access to an OKD cluster using an account with cluster-admin
permissions.
You have installed the OpenShift CLI (oc
).
You have installed your cluster using installer-provisioned infrastructure, or using user-provisioned infrastructure with the platform: none
field set in your install-config.yaml
file.
You have configured hybrid networking with OVN-Kubernetes for your cluster. For more information, see Configuring hybrid networking.
You are running an OKD cluster version 4.6.8 or later.
Windows instances deployed by the WMCO are configured with the containerd container runtime. Because WMCO installs and manages the runtime, it is recommended that you do not manually install containerd on nodes. |
For the comprehensive prerequisites for the Windows Machine Config Operator, see Windows Machine Config Operator prerequisites.
You can install the Windows Machine Config Operator using either the web console or OpenShift CLI (oc
).
Due to a limitation within the Windows operating system, |
You can use the OKD web console to install the Windows Machine Config Operator (WMCO).
Dual NIC is not supported on WMCO-managed Windows instances. |
From the Administrator perspective in the OKD web console, navigate to the Operators → OperatorHub page.
Use the Filter by keyword box to search for Windows Machine Config Operator
in the catalog. Click the Windows Machine Config Operator tile.
Review the information about the Operator and click Install.
On the Install Operator page:
Select the stable channel as the Update Channel. The stable channel enables the latest stable release of the WMCO to be installed.
The Installation Mode is preconfigured because the WMCO must be available in a single namespace only.
Choose the Installed Namespace for the WMCO. The default Operator recommended namespace is openshift-windows-machine-config-operator
.
Click the Enable Operator recommended cluster monitoring on the Namespace checkbox to enable cluster monitoring for the WMCO.
Select an Approval Strategy.
The Automatic strategy allows Operator Lifecycle Manager (OLM) to automatically update the Operator when a new version is available.
The Manual strategy requires a user with appropriate credentials to approve the Operator update.
Click Install. The WMCO is now listed on the Installed Operators page.
The WMCO is installed automatically into the namespace you defined, like |
Verify that the Status shows Succeeded to confirm successful installation of the WMCO.
You can use the OpenShift CLI (oc
) to install the Windows Machine Config Operator (WMCO).
Dual NIC is not supported on WMCO-managed Windows instances. |
Create a namespace for the WMCO.
Create a Namespace
object YAML file for the WMCO. For example, wmco-namespace.yaml
:
apiVersion: v1
kind: Namespace
metadata:
name: openshift-windows-machine-config-operator (1)
labels:
openshift.io/cluster-monitoring: "true" (2)
1 | It is recommended to deploy the WMCO in the openshift-windows-machine-config-operator namespace. |
2 | This label is required for enabling cluster monitoring for the WMCO. |
Create the namespace:
$ oc create -f <file-name>.yaml
For example:
$ oc create -f wmco-namespace.yaml
Create the Operator group for the WMCO.
Create an OperatorGroup
object YAML file. For example, wmco-og.yaml
:
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
name: windows-machine-config-operator
namespace: openshift-windows-machine-config-operator
spec:
targetNamespaces:
- openshift-windows-machine-config-operator
Create the Operator group:
$ oc create -f <file-name>.yaml
For example:
$ oc create -f wmco-og.yaml
Subscribe the namespace to the WMCO.
Create a Subscription
object YAML file. For example, wmco-sub.yaml
:
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
name: windows-machine-config-operator
namespace: openshift-windows-machine-config-operator
spec:
channel: "stable" (1)
installPlanApproval: "Automatic" (2)
name: "windows-machine-config-operator"
source: "redhat-operators" (3)
sourceNamespace: "openshift-marketplace" (4)
1 | Specify stable as the channel. |
2 | Set an approval strategy. You can set Automatic or Manual . |
3 | Specify the redhat-operators catalog source, which contains the windows-machine-config-operator package manifests. If your OKD is installed on a restricted network, also known as a disconnected cluster, specify the name of the CatalogSource object you created when you configured the Operator LifeCycle Manager (OLM). |
4 | Namespace of the catalog source. Use openshift-marketplace for the default OperatorHub catalog sources. |
Create the subscription:
$ oc create -f <file-name>.yaml
For example:
$ oc create -f wmco-sub.yaml
The WMCO is now installed to the openshift-windows-machine-config-operator
.
Verify the WMCO installation:
$ oc get csv -n openshift-windows-machine-config-operator
NAME DISPLAY VERSION REPLACES PHASE
windows-machine-config-operator.2.0.0 Windows Machine Config Operator 2.0.0 Succeeded
To run the Windows Machine Config Operator (WMCO), you must create a secret in the WMCO namespace containing a private key. This is required to allow the WMCO to communicate with the Windows virtual machine (VM).
You installed the Windows Machine Config Operator (WMCO) using Operator Lifecycle Manager (OLM).
You created a PEM-encoded file containing an RSA key.
Define the secret required to access the Windows VMs:
$ oc create secret generic cloud-private-key --from-file=private-key.pem=${HOME}/.ssh/<key> \
-n openshift-windows-machine-config-operator (1)
1 | You must create the private key in the WMCO namespace, like openshift-windows-machine-config-operator . |
It is recommended to use a different private key than the one used when installing the cluster.
The Windows Machine Config Operator (WMCO) can consume and use a cluster-wide egress proxy configuration when making external requests outside the cluster’s internal network.
This allows you to add Windows nodes and run workloads in a proxy-enabled cluster, allowing your Windows nodes to pull images from registries that are secured behind your proxy server or to make requests to off-cluster services and services that use a custom public key infrastructure.
The cluster-wide proxy affects system components only, not user workloads. |
In proxy-enabled clusters, the WMCO is aware of the NO_PROXY
, HTTP_PROXY
, and HTTPS_PROXY
values that are set for the cluster. The WMCO periodically checks whether the proxy environment variables have changed. If there is a discrepancy, the WMCO reconciles and updates the proxy environment variables on the Windows instances.
Windows workloads created on Windows nodes in proxy-enabled clusters do not inherit proxy settings from the node by default, the same as with Linux nodes. Also, by default PowerShell sessions do not inherit proxy settings on Windows nodes in proxy-enabled clusters.
The Windows Machine Config Operator (WMCO) can pull images from a registry mirror rather than from a public registry by using an ImageDigestMirrorSet
(IDMS) or ImageTagMirrorSet
(ITMS) object to configure your cluster to pull images from the mirror registry.
A mirror registry has the following benefits:
Avoids public registry outages
Speeds up node and pod creation
Pulls images from behind your organization’s firewall
A mirror registry can also be used with a OKD cluster in a disconnected, or air-gapped, network. A disconnected network is a restricted network without direct internet connectivity. Because the cluster does not have access to the internet, any external container images cannot be referenced.
Using a mirror registry requires the following general steps:
Create the mirror registry, using a tool such as Red Hat Quay.
Create a container image registry credentials file.
Copy the images from your online image repository to your mirror registry.
For information about these steps, see "About disconnected installation mirroring."
After creating the mirror registry and mirroring the images, you can use an ImageDigestMirrorSet
(IDMS) or ImageTagMirrorSet
(ITMS) object to configure your cluster to pull images from the mirror registry without needing to update each of your pod specs. The IDMS and ITMS objects redirect requests to pull images from a repository on a source image registry and have it resolved by the mirror repository instead.
If changes are made to the IDMS or ITMS object, the WMCO automatically updates the appropriate hosts.toml
file on your Windows nodes with the new information. Note that the WMCO sequentially updates each Windows node when mirror settings are changed. As such, the time required for these updates increases with the number of Windows nodes in the cluster.
Also, because Windows nodes configured by the WMCO rely on containerd container runtime, the WMCO ensures that the containerd config files are up-to-date with the registry settings. For new nodes, these files are copied to the instances upon creation. For existing nodes, after activating the mirror registry, the registry controller uses SSH to access each node and copy the generated config files, replacing any existing files.
You can use a mirror registry with machine set or Bring-Your-Own-Host (BYOH) Windows nodes.
Setting up container registry repository mirroring enables you to perform the following tasks:
Configure your OKD cluster to redirect requests to pull images from a repository on a source image registry and have it resolved by a repository on a mirrored image registry.
Identify multiple mirrored repositories for each target repository, to make sure that if one mirror is down, another can be used.
Repository mirroring in OKD includes the following attributes:
Image pulls are resilient to registry downtimes.
Clusters in disconnected environments can pull images from critical locations, such as quay.io, and have registries behind a company firewall provide the requested images.
A particular order of registries is tried when an image pull request is made, with the permanent registry typically being the last one tried.
The mirror information you enter is added to the appropriate hosts.toml
containerd configuration file(s) on every Windows node in the OKD cluster.
When a node makes a request for an image from the source repository, it tries each mirrored repository in turn until it finds the requested content. If all mirrors fail, the cluster tries the source repository. If successful, the image is pulled to the node.
Setting up repository mirroring can be done in the following ways:
At OKD installation:
By pulling container images needed by OKD and then bringing those images behind your company’s firewall, you can install OKD into a data center that is in a disconnected environment.
After OKD installation:
If you did not configure mirroring during OKD installation, you can do so postinstallation by using any of the following custom resource (CR) objects:
ImageDigestMirrorSet
(IDMS). This object allows you to pull images from a mirrored registry by using digest specifications. The IDMS CR enables you to set a fall back policy that allows or stops continued attempts to pull from the source registry if the image pull fails.
ImageTagMirrorSet
(ITMS). This object allows you to pull images from a mirrored registry by using image tags. The ITMS CR enables you to set a fall back policy that allows or stops continued attempts to pull from the source registry if the image pull fails.
Each of these custom resource objects identify the following information:
The source of the container image repository you want to mirror.
A separate entry for each mirror repository you want to offer the content requested from the source repository.
The Windows Machine Config Operator (WMCO) watches for changes to the IDMS and ITMS resources and generates a set of hosts.toml
containerd configuration files, one file for each source registry, with those changes. The WMCO then updates any existing Windows nodes to use the new registry configuration.
The IDMS and ITMS objects must be created before you can add Windows nodes using a mirrored registry. |
You can create postinstallation mirror configuration custom resources (CR) to redirect image pull requests from a source image registry to a mirrored image registry.
Windows images mirrored through |
Access to the cluster as a user with the cluster-admin
role.
Configure mirrored repositories, by either:
Setting up a mirrored repository with Red Hat Quay, as described in Red Hat Quay Repository Mirroring. Using Red Hat Quay allows you to copy images from one repository to another and also automatically sync those repositories repeatedly over time.
Using a tool such as skopeo
to copy images manually from the source repository to the mirrored repository.
For example, after installing the skopeo RPM package on a Red Hat Enterprise Linux (RHEL) 7 or RHEL 8 system, use the skopeo
command as shown in this example:
$ skopeo copy --all \
docker://registry.access.redhat.com/ubi9/ubi-minimal:latest@sha256:5cf... \
docker://example.io/example/ubi-minimal
In this example, you have a container image registry that is named example.io
with an image repository named example
to which you want to copy the ubi9/ubi-minimal
image from registry.access.redhat.com
. After you create the mirrored registry, you can configure your OKD cluster to redirect requests made of the source repository to the mirrored repository.
You must mirror the
|
Log in to your OKD cluster.
Create an ImageDigestMirrorSet
or ImageTagMirrorSet
CR, as needed, replacing the source and mirrors with your own registry and repository pairs and images:
apiVersion: config.openshift.io/v1 (1)
kind: ImageDigestMirrorSet (2)
metadata:
name: ubi9repo
spec:
imageDigestMirrors: (3)
- mirrors:
- example.io/example/ubi-minimal (4)
- example.com/example2/ubi-minimal (5)
source: registry.access.redhat.com/ubi9/ubi-minimal (6)
mirrorSourcePolicy: AllowContactingSource (7)
- mirrors:
- mirror.example.com
source: registry.redhat.io
mirrorSourcePolicy: NeverContactSource
- mirrors:
- docker.io
source: docker-mirror.internal
mirrorSourcePolicy: AllowContactingSource
1 | Indicates the API to use with this CR. This must be config.openshift.io/v1 . |
2 | Indicates the kind of object according to the pull type:
|
3 | Indicates the type of image pull method, either:
|
4 | Indicates the name of the mirrored image registry and repository. |
5 | Optional: Indicates a secondary mirror repository for each target repository. If one mirror is down, the target repository can use another mirror. |
6 | Indicates the registry and repository source, which is the repository that is referred to in image pull specifications. |
7 | Optional: Indicates the fallback policy if the image pull fails:
|
Create the new object:
$ oc create -f registryrepomirror.yaml
To check that the mirrored configuration settings are applied, do the following on one of the nodes.
List your nodes:
$ oc get node
NAME STATUS ROLES AGE VERSION
ip-10-0-137-44.ec2.internal Ready worker 7m v1.30.3
ip-10-0-138-148.ec2.internal Ready master 11m v1.30.3
ip-10-0-139-122.ec2.internal Ready master 11m v1.30.3
ip-10-0-147-35.ec2.internal Ready worker 7m v1.30.3
ip-10-0-153-12.ec2.internal Ready worker 7m v1.30.3
ip-10-0-154-10.ec2.internal Ready master 11m v1.30.3
Start the debugging process to access the node:
$ oc debug node/ip-10-0-147-35.ec2.internal
Starting pod/ip-10-0-147-35ec2internal-debug ...
To use host binaries, run `chroot /host`
Change your root directory to /host
:
sh-4.2# chroot /host
Check that the WMCO generated a hosts.toml
file for each registry on each Windows instance. For the previous example IDMS object, there should be three files in the following file structure:
$ tree $config_path
C:/k/containerd/registries/
|── registry.access.redhat.com
| └── hosts.toml
|── mirror.example.com
| └── hosts.toml
└── docker.io
└── hosts.toml:
The following output represents a hosts.toml
containerd configuration file where the previous example IDMS object was applied.
$ cat "$config_path"/registry.access.redhat.com/host.toml
server = "https://registry.access.redhat.com" # default fallback server since "AllowContactingSource" mirrorSourcePolicy is set
[host."https://example.io/example/ubi-minimal"]
capabilities = ["pull"]
[host."https://example.com/example2/ubi-minimal"] # secondary mirror
capabilities = ["pull"]
$ cat "$config_path"/registry.redhat.io/host.toml
# "server" omitted since "NeverContactSource" mirrorSourcePolicy is set
[host."https://mirror.example.com"]
capabilities = ["pull"]
$ cat "$config_path"/docker.io/host.toml
server = "https://docker.io"
[host."https://docker-mirror.internal"]
capabilities = ["pull", "resolve"] # resolve tags
Pull an image to the node from the source and check if it is resolved by the mirror.
sh-4.2# podman pull --log-level=debug registry.access.redhat.com/ubi9/ubi-minimal@sha256:5cf...
If the repository mirroring procedure does not work as described, use the following information about how repository mirroring works to help troubleshoot the problem.
The first working mirror is used to supply the pulled image.
The main registry is only used if no other mirror works.
From the system context, the Insecure
flags are used as fallback.