# podman pull quay.io/openshift-kni/telco-ran-tools:latest
In environments with limited bandwidth where you use the GitOps Zero Touch Provisioning (ZTP) solution to deploy a large number of clusters, you want to avoid downloading all the images that are required for bootstrapping and installing OKD. The limited bandwidth at remote single-node OpenShift sites can cause long deployment times. The factory-precaching-cli tool allows you to pre-stage servers before shipping them to the remote site for ZTP provisioning.
The factory-precaching-cli tool does the following:
Downloads the RHCOS rootfs image that is required by the minimal ISO to boot.
Creates a partition from the installation disk labelled as data
.
Formats the disk in xfs.
Creates a GUID Partition Table (GPT) data partition at the end of the disk, where the size of the partition is configurable by the tool.
Copies the container images required to install OKD.
Copies the container images required by ZTP to install OKD.
Optional: Copies Day-2 Operators to the partition.
The factory-precaching-cli tool is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
The factory-precaching-cli tool Go binary is publicly available in the {rds-first} tools container image.
The factory-precaching-cli tool Go binary in the container image is executed on the server running an FCOS live image using podman
.
If you are working in a disconnected environment or have a private registry, you need to copy the image there so you can download the image to the server.
Pull the factory-precaching-cli tool image by running the following command:
# podman pull quay.io/openshift-kni/telco-ran-tools:latest
To check that the tool is available, query the current version of the factory-precaching-cli tool Go binary:
# podman run quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli -v
factory-precaching-cli version 20221018.120852+main.feecf17
You can use the factory-precaching-cli tool with to boot servers where only one disk is available and external disk drive cannot be attached to the server.
FCOS requires the disk to not be in use when the disk is about to be written with an FCOS image. |
Depending on the server hardware, you can mount the FCOS live ISO on the blank server using one of the following methods:
Using the Dell RACADM tool on a Dell server.
Using the HPONCFG tool on a HP server.
Using the Redfish BMC API.
It is recommended to automate the mounting procedure. To automate the procedure, you need to pull the required images and host them on a local HTTP server. |
You powered up the host.
You have network connectivity to the host.
Procedure
This example procedure uses the Redfish BMC API to mount the FCOS live ISO. |
Mount the FCOS live ISO:
Check virtual media status:
$ curl --globoff -H "Content-Type: application/json" -H \
"Accept: application/json" -k -X GET --user ${username_password} \
https://$BMC_ADDRESS/redfish/v1/Managers/Self/VirtualMedia/1 | python -m json.tool
Mount the ISO file as a virtual media:
$ curl --globoff -L -w "%{http_code} %{url_effective}\\n" -ku ${username_password} -H "Content-Type: application/json" -H "Accept: application/json" -d '{"Image": "http://[$HTTPd_IP]/RHCOS-live.iso"}' -X POST https://$BMC_ADDRESS/redfish/v1/Managers/Self/VirtualMedia/1/Actions/VirtualMedia.InsertMedia
Set the boot order to boot from the virtual media once:
$ curl --globoff -L -w "%{http_code} %{url_effective}\\n" -ku ${username_password} -H "Content-Type: application/json" -H "Accept: application/json" -d '{"Boot":{ "BootSourceOverrideEnabled": "Once", "BootSourceOverrideTarget": "Cd", "BootSourceOverrideMode": "UEFI"}}' -X PATCH https://$BMC_ADDRESS/redfish/v1/Systems/Self
Reboot and ensure that the server is booting from virtual media.
For more information about the butane
utility, see About Butane.
For more information about creating a custom live FCOS ISO, see Creating a custom live FCOS ISO for remote server access.
For more information about using the Dell RACADM tool, see Integrated Dell Remote Access Controller 9 RACADM CLI Guide.
For more information about using the HP HPONCFG tool, see Using HPONCFG.
For more information about using the Redfish BMC API, see Booting from an HTTP-hosted ISO image using the Redfish API.
To run the full pre-caching process, you have to boot from a live ISO and use the factory-precaching-cli tool from a container image to partition and pre-cache all the artifacts required.
A live ISO or FCOS live ISO is required because the disk must not be in use when the operating system (FCOS) is written to the device during the provisioning. Single-disk servers can also be enabled with this procedure.
You have a disk that is not partitioned.
You have access to the quay.io/openshift-kni/telco-ran-tools:latest
image.
You have enough storage to install OKD and pre-cache the required images.
Verify that the disk is cleared:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 93.8G 0 loop /run/ephemeral
loop1 7:1 0 897.3M 1 loop /sysroot
sr0 11:0 1 999M 0 rom /run/media/iso
nvme0n1 259:1 0 1.5T 0 disk
Erase any file system, RAID or partition table signatures from the device:
# wipefs -a /dev/nvme0n1
/dev/nvme0n1: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 8 bytes were erased at offset 0x1749a955e00 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
The tool fails if the disk is not empty because it uses partition number 1 of the device for pre-caching the artifacts. |
Once the device is ready, you create a single partition and a GPT partition table.
The partition is automatically labelled as data
and created at the end of the device.
Otherwise, the partition will be overridden by the coreos-installer
.
The |
The container must run as privileged
due to formatting host devices.
You have to mount the /dev
folder so that the process can be executed inside the container.
In the following example, the size of the partition is 250 GiB due to allow pre-caching the DU profile for Day 2 Operators.
Run the container as privileged
and partition the disk:
# podman run -v /dev:/dev --privileged \
--rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli partition \ (1)
-d /dev/nvme0n1 \ (2)
-s 250 (3)
1 | Specifies the partitioning function of the factory-precaching-cli tool. |
2 | Defines the root directory on the disk. |
3 | Defines the size of the disk in GB. |
Check the storage information:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 93.8G 0 loop /run/ephemeral
loop1 7:1 0 897.3M 1 loop /sysroot
sr0 11:0 1 999M 0 rom /run/media/iso
nvme0n1 259:1 0 1.5T 0 disk
└─nvme0n1p1 259:3 0 250G 0 part
You must verify that the following requirements are met:
The device has a GPT partition table
The partition uses the latest sectors of the device.
The partition is correctly labeled as data
.
Query the disk status to verify that the disk is partitioned as expected:
# gdisk -l /dev/nvme0n1
GPT fdisk (gdisk) version 1.0.3
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.
Disk /dev/nvme0n1: 3125627568 sectors, 1.5 TiB
Model: Dell Express Flash PM1725b 1.6TB SFF
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): CB5A9D44-9B3C-4174-A5C1-C64957910B61
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 3125627534
Partitions will be aligned on 2048-sector boundaries
Total free space is 2601338846 sectors (1.2 TiB)
Number Start (sector) End (sector) Size Code Name
1 2601338880 3125627534 250.0 GiB 8300 data
After verifying that the disk is partitioned correctly, you can mount the device into /mnt
.
It is recommended to mount the device into |
Verify that the partition is formatted as xfs
:
# lsblk -f /dev/nvme0n1
NAME FSTYPE LABEL UUID MOUNTPOINT
nvme0n1
└─nvme0n1p1 xfs 1bee8ea4-d6cf-4339-b690-a76594794071
Mount the partition:
# mount /dev/nvme0n1p1 /mnt/
Check that the partition is mounted:
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 93.8G 0 loop /run/ephemeral
loop1 7:1 0 897.3M 1 loop /sysroot
sr0 11:0 1 999M 0 rom /run/media/iso
nvme0n1 259:1 0 1.5T 0 disk
└─nvme0n1p1 259:2 0 250G 0 part /var/mnt (1)
1 | The mount point is /var/mnt because the /mnt folder in FCOS is a link to /var/mnt . |
The factory-precaching-cli tool allows you to download the following images to your partitioned server:
OKD images
Operator images that are included in the distributed unit (DU) profile for 5G RAN sites
Operator images from disconnected registries
The list of available Operator images can vary in different OKD releases. |
The factory-precaching-cli tool uses parallel workers to download multiple images simultaneously.
You can configure the number of workers with the --parallel
or -p
option.
The default number is set to 80% of the available CPUs to the server.
Your login shell may be restricted to a subset of CPUs, which reduces the CPUs available to the container.
To remove this restriction, you can precede your commands with
|
To download OKD container images, you need to know the multicluster engine version. When you use the --du-profile
flag, you also need to specify the Red Hat Advanced Cluster Management (RHACM) version running in the hub cluster that is going to provision the single-node OpenShift.
You have RHACM and the multicluster engine Operator installed.
You partitioned the storage device.
You have enough space for the images on the partitioned device.
You connected the bare-metal server to the Internet.
You have a valid pull secret.
Check the RHACM version and the multicluster engine version by running the following commands in the hub cluster:
$ oc get csv -A | grep -i advanced-cluster-management
open-cluster-management advanced-cluster-management.v2.6.3 Advanced Cluster Management for Kubernetes 2.6.3 advanced-cluster-management.v2.6.3 Succeeded
$ oc get csv -A | grep -i multicluster-engine
multicluster-engine cluster-group-upgrades-operator.v0.0.3 cluster-group-upgrades-operator 0.0.3 Pending
multicluster-engine multicluster-engine.v2.1.4 multicluster engine for Kubernetes 2.1.4 multicluster-engine.v2.0.3 Succeeded
multicluster-engine openshift-gitops-operator.v1.5.7 Red Hat OpenShift GitOps 1.5.7 openshift-gitops-operator.v1.5.6-0.1664915551.p Succeeded
multicluster-engine openshift-pipelines-operator-rh.v1.6.4 Red Hat OpenShift Pipelines 1.6.4 openshift-pipelines-operator-rh.v1.6.3 Succeeded
To access the container registry, copy a valid pull secret on the server to be installed:
Create the .docker
folder:
$ mkdir /root/.docker
Copy the valid pull in the config.json
file to the previously created .docker/
folder:
$ cp config.json /root/.docker/config.json (1)
1 | /root/.docker/config.json is the default path where podman checks for the login credentials for the registry. |
If you use a different registry to pull the required artifacts, you need to copy the proper pull secret. If the local registry uses TLS, you need to include the certificates from the registry as well. |
The factory-precaching-cli tool allows you to pre-cache all the container images required to provision a specific OKD release.
Pre-cache the release by running the following command:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools -- \
factory-precaching-cli download \ (1)
-r 4.17.0 \ (2)
--acm-version 2.6.3 \ (3)
--mce-version 2.1.4 \ (4)
-f /mnt \ (5)
--img quay.io/custom/repository (6)
1 | Specifies the downloading function of the factory-precaching-cli tool. |
2 | Defines the OKD release version. |
3 | Defines the RHACM version. |
4 | Defines the multicluster engine version. |
5 | Defines the folder where you want to download the images on the disk. |
6 | Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk. |
Generated /mnt/imageset.yaml
Generating list of pre-cached artifacts...
Processing artifact [1/176]: ocp-v4.0-art-dev@sha256_6ac2b96bf4899c01a87366fd0feae9f57b1b61878e3b5823da0c3f34f707fbf5
Processing artifact [2/176]: ocp-v4.0-art-dev@sha256_f48b68d5960ba903a0d018a10544ae08db5802e21c2fa5615a14fc58b1c1657c
Processing artifact [3/176]: ocp-v4.0-art-dev@sha256_a480390e91b1c07e10091c3da2257180654f6b2a735a4ad4c3b69dbdb77bbc06
Processing artifact [4/176]: ocp-v4.0-art-dev@sha256_ecc5d8dbd77e326dba6594ff8c2d091eefbc4d90c963a9a85b0b2f0e6155f995
Processing artifact [5/176]: ocp-v4.0-art-dev@sha256_274b6d561558a2f54db08ea96df9892315bb773fc203b1dbcea418d20f4c7ad1
Processing artifact [6/176]: ocp-v4.0-art-dev@sha256_e142bf5020f5ca0d1bdda0026bf97f89b72d21a97c9cc2dc71bf85050e822bbf
...
Processing artifact [175/176]: ocp-v4.0-art-dev@sha256_16cd7eda26f0fb0fc965a589e1e96ff8577e560fcd14f06b5fda1643036ed6c8
Processing artifact [176/176]: ocp-v4.0-art-dev@sha256_cf4d862b4a4170d4f611b39d06c31c97658e309724f9788e155999ae51e7188f
...
Summary:
Release: 4.17.0
Hub Version: 2.6.3
ACM Version: 2.6.3
MCE Version: 2.1.4
Include DU Profile: No
Workers: 83
Check that all the images are compressed in the target folder of server:
$ ls -l /mnt (1)
1 | It is recommended that you pre-cache the images in the /mnt folder. |
-rw-r--r--. 1 root root 136352323 Oct 31 15:19 ocp-v4.0-art-dev@sha256_edec37e7cd8b1611d0031d45e7958361c65e2005f145b471a8108f1b54316c07.tgz
-rw-r--r--. 1 root root 156092894 Oct 31 15:33 ocp-v4.0-art-dev@sha256_ee51b062b9c3c9f4fe77bd5b3cc9a3b12355d040119a1434425a824f137c61a9.tgz
-rw-r--r--. 1 root root 172297800 Oct 31 15:29 ocp-v4.0-art-dev@sha256_ef23d9057c367a36e4a5c4877d23ee097a731e1186ed28a26c8d21501cd82718.tgz
-rw-r--r--. 1 root root 171539614 Oct 31 15:23 ocp-v4.0-art-dev@sha256_f0497bb63ef6834a619d4208be9da459510df697596b891c0c633da144dbb025.tgz
-rw-r--r--. 1 root root 160399150 Oct 31 15:20 ocp-v4.0-art-dev@sha256_f0c339da117cde44c9aae8d0bd054bceb6f19fdb191928f6912a703182330ac2.tgz
-rw-r--r--. 1 root root 175962005 Oct 31 15:17 ocp-v4.0-art-dev@sha256_f19dd2e80fb41ef31d62bb8c08b339c50d193fdb10fc39cc15b353cbbfeb9b24.tgz
-rw-r--r--. 1 root root 174942008 Oct 31 15:33 ocp-v4.0-art-dev@sha256_f1dbb81fa1aa724e96dd2b296b855ff52a565fbef003d08030d63590ae6454df.tgz
-rw-r--r--. 1 root root 246693315 Oct 31 15:31 ocp-v4.0-art-dev@sha256_f44dcf2c94e4fd843cbbf9b11128df2ba856cd813786e42e3da1fdfb0f6ddd01.tgz
-rw-r--r--. 1 root root 170148293 Oct 31 15:00 ocp-v4.0-art-dev@sha256_f48b68d5960ba903a0d018a10544ae08db5802e21c2fa5615a14fc58b1c1657c.tgz
-rw-r--r--. 1 root root 168899617 Oct 31 15:16 ocp-v4.0-art-dev@sha256_f5099b0989120a8d08a963601214b5c5cb23417a707a8624b7eb52ab788a7f75.tgz
-rw-r--r--. 1 root root 176592362 Oct 31 15:05 ocp-v4.0-art-dev@sha256_f68c0e6f5e17b0b0f7ab2d4c39559ea89f900751e64b97cb42311a478338d9c3.tgz
-rw-r--r--. 1 root root 157937478 Oct 31 15:37 ocp-v4.0-art-dev@sha256_f7ba33a6a9db9cfc4b0ab0f368569e19b9fa08f4c01a0d5f6a243d61ab781bd8.tgz
-rw-r--r--. 1 root root 145535253 Oct 31 15:26 ocp-v4.0-art-dev@sha256_f8f098911d670287826e9499806553f7a1dd3e2b5332abbec740008c36e84de5.tgz
-rw-r--r--. 1 root root 158048761 Oct 31 15:40 ocp-v4.0-art-dev@sha256_f914228ddbb99120986262168a705903a9f49724ffa958bb4bf12b2ec1d7fb47.tgz
-rw-r--r--. 1 root root 167914526 Oct 31 15:37 ocp-v4.0-art-dev@sha256_fa3ca9401c7a9efda0502240aeb8d3ae2d239d38890454f17fe5158b62305010.tgz
-rw-r--r--. 1 root root 164432422 Oct 31 15:24 ocp-v4.0-art-dev@sha256_fc4783b446c70df30b3120685254b40ce13ba6a2b0bf8fb1645f116cf6a392f1.tgz
-rw-r--r--. 1 root root 306643814 Oct 31 15:11 troubleshoot@sha256_b86b8aea29a818a9c22944fd18243fa0347c7a2bf1ad8864113ff2bb2d8e0726.tgz
You can also pre-cache Day-2 Operators used in the 5G Radio Access Network (RAN) Distributed Unit (DU) cluster configuration. The Day-2 Operators depend on the installed OKD version.
You need to include the RHACM hub and multicluster engine Operator versions by using the |
Pre-cache the Operator images:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download \ (1)
-r 4.17.0 \ (2)
--acm-version 2.6.3 \ (3)
--mce-version 2.1.4 \ (4)
-f /mnt \ (5)
--img quay.io/custom/repository (6)
--du-profile -s (7)
1 | Specifies the downloading function of the factory-precaching-cli tool. |
2 | Defines the OKD release version. |
3 | Defines the RHACM version. |
4 | Defines the multicluster engine version. |
5 | Defines the folder where you want to download the images on the disk. |
6 | Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk. |
7 | Specifies pre-caching the Operators included in the DU configuration. |
Generated /mnt/imageset.yaml
Generating list of pre-cached artifacts...
Processing artifact [1/379]: ocp-v4.0-art-dev@sha256_7753a8d9dd5974be8c90649aadd7c914a3d8a1f1e016774c7ac7c9422e9f9958
Processing artifact [2/379]: ose-kube-rbac-proxy@sha256_c27a7c01e5968aff16b6bb6670423f992d1a1de1a16e7e260d12908d3322431c
Processing artifact [3/379]: ocp-v4.0-art-dev@sha256_370e47a14c798ca3f8707a38b28cfc28114f492bb35fe1112e55d1eb51022c99
...
Processing artifact [378/379]: ose-local-storage-operator@sha256_0c81c2b79f79307305e51ce9d3837657cf9ba5866194e464b4d1b299f85034d0
Processing artifact [379/379]: multicluster-operators-channel-rhel8@sha256_c10f6bbb84fe36e05816e873a72188018856ad6aac6cc16271a1b3966f73ceb3
...
Summary:
Release: 4.17.0
Hub Version: 2.6.3
ACM Version: 2.6.3
MCE Version: 2.1.4
Include DU Profile: Yes
Workers: 83
The --generate-imageset
argument stops the factory-precaching-cli tool after the ImageSetConfiguration
custom resource (CR) is generated.
This allows you to customize the ImageSetConfiguration
CR before downloading any images.
After you customized the CR, you can use the --skip-imageset
argument to download the images that you specified in the ImageSetConfiguration
CR.
You can customize the ImageSetConfiguration
CR in the following ways:
Add Operators and additional images
Remove Operators and additional images
Change Operator and catalog sources to local or disconnected registries
Pre-cache the images:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download \ (1)
-r 4.17.0 \ (2)
--acm-version 2.6.3 \ (3)
--mce-version 2.1.4 \ (4)
-f /mnt \ (5)
--img quay.io/custom/repository (6)
--du-profile -s \ (7)
--generate-imageset (8)
1 | Specifies the downloading function of the factory-precaching-cli tool. |
2 | Defines the OKD release version. |
3 | Defines the RHACM version. |
4 | Defines the multicluster engine version. |
5 | Defines the folder where you want to download the images on the disk. |
6 | Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk. |
7 | Specifies pre-caching the Operators included in the DU configuration. |
8 | The --generate-imageset argument generates the ImageSetConfiguration CR only, which allows you to customize the CR. |
Generated /mnt/imageset.yaml
apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
mirror:
platform:
channels:
- name: stable-4.17
minVersion: 4.17.0 (1)
maxVersion: 4.17.0
additionalImages:
- name: quay.io/custom/repository
operators:
- catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
packages:
- name: advanced-cluster-management (2)
channels:
- name: 'release-2.6'
minVersion: 2.6.3
maxVersion: 2.6.3
- name: multicluster-engine (2)
channels:
- name: 'stable-2.1'
minVersion: 2.1.4
maxVersion: 2.1.4
- name: local-storage-operator (3)
channels:
- name: 'stable'
- name: ptp-operator (3)
channels:
- name: 'stable'
- name: sriov-network-operator (3)
channels:
- name: 'stable'
- name: cluster-logging (3)
channels:
- name: 'stable'
- name: lvms-operator (3)
channels:
- name: 'stable-4.17'
- name: amq7-interconnect-operator (3)
channels:
- name: '1.10.x'
- name: bare-metal-event-relay (3)
channels:
- name: 'stable'
- catalog: registry.redhat.io/redhat/certified-operator-index:v4.17
packages:
- name: sriov-fec (3)
channels:
- name: 'stable'
1 | The platform versions match the versions passed to the tool. |
2 | The versions of RHACM and the multicluster engine Operator match the versions passed to the tool. |
3 | The CR contains all the specified DU Operators. |
Customize the catalog resource in the CR:
apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
mirror:
platform:
[...]
operators:
- catalog: eko4.cloud.lab.eng.bos.redhat.com:8443/redhat/certified-operator-index:v4.17
packages:
- name: sriov-fec
channels:
- name: 'stable'
When you download images by using a local or disconnected registry, you have to first add certificates for the registries that you want to pull the content from.
To avoid any errors, copy the registry certificate into your server:
# cp /tmp/eko4-ca.crt /etc/pki/ca-trust/source/anchors/.
Then, update the certificates trust store:
# update-ca-trust
Mount the host /etc/pki
folder into the factory-cli image:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker -v /etc/pki:/etc/pki --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli download \ (1)
-r 4.17.0 \ (2)
--acm-version 2.6.3 \ (3)
--mce-version 2.1.4 \ (4)
-f /mnt \ (5)
--img quay.io/custom/repository (6)
--du-profile -s \ (7)
--skip-imageset (8)
1 | Specifies the downloading function of the factory-precaching-cli tool. |
2 | Defines the OKD release version. |
3 | Defines the RHACM version. |
4 | Defines the multicluster engine version. |
5 | Defines the folder where you want to download the images on the disk. |
6 | Optional. Defines the repository where you store your additional images. These images are downloaded and pre-cached on the disk. |
7 | Specifies pre-caching the Operators included in the DU configuration. |
8 | The --skip-imageset argument allows you to download the images that you specified in your customized ImageSetConfiguration CR. |
Download the images without generating a new imageSetConfiguration
CR:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker --privileged --rm quay.io/openshift-kni/telco-ran-tools:latest -- factory-precaching-cli download -r 4.17.0 \
--acm-version 2.6.3 --mce-version 2.1.4 -f /mnt \
--img quay.io/custom/repository \
--du-profile -s \
--skip-imageset
To access the online Red Hat registries, see OpenShift installation customization tools.
For more information about using the multicluster engine, see About cluster lifecycle with the multicluster engine operator.
The SiteConfig
manifest defines how an OpenShift cluster is to be installed and configured.
In the GitOps Zero Touch Provisioning (ZTP) provisioning workflow, the factory-precaching-cli tool requires the following additional fields in the SiteConfig
manifest:
clusters.ignitionConfigOverride
nodes.installerArgs
nodes.ignitionConfigOverride
apiVersion: ran.openshift.io/v1
kind: SiteConfig
metadata:
name: "example-5g-lab"
namespace: "example-5g-lab"
spec:
baseDomain: "example.domain.redhat.com"
pullSecretRef:
name: "assisted-deployment-pull-secret"
clusterImageSetNameRef: "img4.9.10-x86-64-appsub" (1)
sshPublicKey: "ssh-rsa ..."
clusters:
- clusterName: "sno-worker-0"
clusterImageSetNameRef: "eko4-img4.11.5-x86-64-appsub" (2)
clusterLabels:
group-du-sno: ""
common-411: true
sites : "example-5g-lab"
vendor: "OpenShift"
clusterNetwork:
- cidr: 10.128.0.0/14
hostPrefix: 23
machineNetwork:
- cidr: 10.19.32.192/26
serviceNetwork:
- 172.30.0.0/16
networkType: "OVNKubernetes"
additionalNTPSources:
- clock.corp.redhat.com
ignitionConfigOverride:
'{
"ignition": {
"version": "3.1.0"
},
"systemd": {
"units": [
{
"name": "var-mnt.mount",
"enabled": true,
"contents": "[Unit]\nDescription=Mount partition with artifacts\nBefore=precache-images.service\nBindsTo=precache-images.service\nStopWhenUnneeded=true\n\n[Mount]\nWhat=/dev/disk/by-partlabel/data\nWhere=/var/mnt\nType=xfs\nTimeoutSec=30\n\n[Install]\nRequiredBy=precache-images.service"
},
{
"name": "precache-images.service",
"enabled": true,
"contents": "[Unit]\nDescription=Extracts the precached images in discovery stage\nAfter=var-mnt.mount\nBefore=agent.service\n\n[Service]\nType=oneshot\nUser=root\nWorkingDirectory=/var/mnt\nExecStart=bash /usr/local/bin/extract-ai.sh\n#TimeoutStopSec=30\n\n[Install]\nWantedBy=multi-user.target default.target\nWantedBy=agent.service"
}
]
},
"storage": {
"files": [
{
"overwrite": true,
"path": "/usr/local/bin/extract-ai.sh",
"mode": 755,
"user": {
"name": "root"
},
"contents": {
"source": "data:,%23%21%2Fbin%2Fbash%0A%0AFOLDER%3D%22%24%7BFOLDER%3A-%24%28pwd%29%7D%22%0AOCP_RELEASE_LIST%3D%22%24%7BOCP_RELEASE_LIST%3A-ai-images.txt%7D%22%0ABINARY_FOLDER%3D%2Fvar%2Fmnt%0A%0Apushd%20%24FOLDER%0A%0Atotal_copies%3D%24%28sort%20-u%20%24BINARY_FOLDER%2F%24OCP_RELEASE_LIST%20%7C%20wc%20-l%29%20%20%23%20Required%20to%20keep%20track%20of%20the%20pull%20task%20vs%20total%0Acurrent_copy%3D1%0A%0Awhile%20read%20-r%20line%3B%0Ado%0A%20%20uri%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%241%7D%27%29%0A%20%20%23tar%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%242%7D%27%29%0A%20%20podman%20image%20exists%20%24uri%0A%20%20if%20%5B%5B%20%24%3F%20-eq%200%20%5D%5D%3B%20then%0A%20%20%20%20%20%20echo%20%22Skipping%20existing%20image%20%24tar%22%0A%20%20%20%20%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20%20%20%20%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%0A%20%20%20%20%20%20continue%0A%20%20fi%0A%20%20tar%3D%24%28echo%20%22%24uri%22%20%7C%20%20rev%20%7C%20cut%20-d%20%22%2F%22%20-f1%20%7C%20rev%20%7C%20tr%20%22%3A%22%20%22_%22%29%0A%20%20tar%20zxvf%20%24%7Btar%7D.tgz%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-f%20%24%7Btar%7D.gz%3B%20fi%0A%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20skopeo%20copy%20dir%3A%2F%2F%24%28pwd%29%2F%24%7Btar%7D%20containers-storage%3A%24%7Buri%7D%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-rf%20%24%7Btar%7D%3B%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%3B%20fi%0Adone%20%3C%20%24%7BBINARY_FOLDER%7D%2F%24%7BOCP_RELEASE_LIST%7D%0A%0A%23%20workaround%20while%20https%3A%2F%2Fgithub.com%2Fopenshift%2Fassisted-service%2Fpull%2F3546%0A%23cp%20%2Fvar%2Fmnt%2Fmodified-rhcos-4.10.3-x86_64-metal.x86_64.raw.gz%20%2Fvar%2Ftmp%2F.%0A%0Aexit%200"
}
},
{
"overwrite": true,
"path": "/usr/local/bin/agent-fix-bz1964591",
"mode": 755,
"user": {
"name": "root"
},
"contents": {
"source": "data:,%23%21%2Fusr%2Fbin%2Fsh%0A%0A%23%20This%20script%20is%20a%20workaround%20for%20bugzilla%201964591%20where%20symlinks%20inside%20%2Fvar%2Flib%2Fcontainers%2F%20get%0A%23%20corrupted%20under%20some%20circumstances.%0A%23%0A%23%20In%20order%20to%20let%20agent.service%20start%20correctly%20we%20are%20checking%20here%20whether%20the%20requested%0A%23%20container%20image%20exists%20and%20in%20case%20%22podman%20images%22%20returns%20an%20error%20we%20try%20removing%20the%20faulty%0A%23%20image.%0A%23%0A%23%20In%20such%20a%20scenario%20agent.service%20will%20detect%20the%20image%20is%20not%20present%20and%20pull%20it%20again.%20In%20case%0A%23%20the%20image%20is%20present%20and%20can%20be%20detected%20correctly%2C%20no%20any%20action%20is%20required.%0A%0AIMAGE%3D%24%28echo%20%241%20%7C%20sed%20%27s%2F%3A.%2A%2F%2F%27%29%0Apodman%20image%20exists%20%24IMAGE%20%7C%7C%20echo%20%22already%20loaded%22%20%7C%7C%20echo%20%22need%20to%20be%20pulled%22%0A%23podman%20images%20%7C%20grep%20%24IMAGE%20%7C%7C%20podman%20rmi%20--force%20%241%20%7C%7C%20true"
}
}
]
}
}'
nodes:
- hostName: "snonode.sno-worker-0.example.domain.redhat.com"
role: "master"
bmcAddress: "idrac-virtualmedia+https://10.19.28.53/redfish/v1/Systems/System.Embedded.1"
bmcCredentialsName:
name: "worker0-bmh-secret"
bootMACAddress: "e4:43:4b:bd:90:46"
bootMode: "UEFI"
rootDeviceHints:
deviceName: /dev/disk/by-path/pci-0000:01:00.0-scsi-0:2:0:0
installerArgs: '["--save-partlabel", "data"]'
ignitionConfigOverride: |
{
"ignition": {
"version": "3.1.0"
},
"systemd": {
"units": [
{
"name": "var-mnt.mount",
"enabled": true,
"contents": "[Unit]\nDescription=Mount partition with artifacts\nBefore=precache-ocp-images.service\nBindsTo=precache-ocp-images.service\nStopWhenUnneeded=true\n\n[Mount]\nWhat=/dev/disk/by-partlabel/data\nWhere=/var/mnt\nType=xfs\nTimeoutSec=30\n\n[Install]\nRequiredBy=precache-ocp-images.service"
},
{
"name": "precache-ocp-images.service",
"enabled": true,
"contents": "[Unit]\nDescription=Extracts the precached OCP images into containers storage\nAfter=var-mnt.mount\nBefore=machine-config-daemon-pull.service nodeip-configuration.service\n\n[Service]\nType=oneshot\nUser=root\nWorkingDirectory=/var/mnt\nExecStart=bash /usr/local/bin/extract-ocp.sh\nTimeoutStopSec=60\n\n[Install]\nWantedBy=multi-user.target"
}
]
},
"storage": {
"files": [
{
"overwrite": true,
"path": "/usr/local/bin/extract-ocp.sh",
"mode": 755,
"user": {
"name": "root"
},
"contents": {
"source": "data:,%23%21%2Fbin%2Fbash%0A%0AFOLDER%3D%22%24%7BFOLDER%3A-%24%28pwd%29%7D%22%0AOCP_RELEASE_LIST%3D%22%24%7BOCP_RELEASE_LIST%3A-ocp-images.txt%7D%22%0ABINARY_FOLDER%3D%2Fvar%2Fmnt%0A%0Apushd%20%24FOLDER%0A%0Atotal_copies%3D%24%28sort%20-u%20%24BINARY_FOLDER%2F%24OCP_RELEASE_LIST%20%7C%20wc%20-l%29%20%20%23%20Required%20to%20keep%20track%20of%20the%20pull%20task%20vs%20total%0Acurrent_copy%3D1%0A%0Awhile%20read%20-r%20line%3B%0Ado%0A%20%20uri%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%241%7D%27%29%0A%20%20%23tar%3D%24%28echo%20%22%24line%22%20%7C%20awk%20%27%7Bprint%242%7D%27%29%0A%20%20podman%20image%20exists%20%24uri%0A%20%20if%20%5B%5B%20%24%3F%20-eq%200%20%5D%5D%3B%20then%0A%20%20%20%20%20%20echo%20%22Skipping%20existing%20image%20%24tar%22%0A%20%20%20%20%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20%20%20%20%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%0A%20%20%20%20%20%20continue%0A%20%20fi%0A%20%20tar%3D%24%28echo%20%22%24uri%22%20%7C%20%20rev%20%7C%20cut%20-d%20%22%2F%22%20-f1%20%7C%20rev%20%7C%20tr%20%22%3A%22%20%22_%22%29%0A%20%20tar%20zxvf%20%24%7Btar%7D.tgz%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-f%20%24%7Btar%7D.gz%3B%20fi%0A%20%20echo%20%22Copying%20%24%7Buri%7D%20%5B%24%7Bcurrent_copy%7D%2F%24%7Btotal_copies%7D%5D%22%0A%20%20skopeo%20copy%20dir%3A%2F%2F%24%28pwd%29%2F%24%7Btar%7D%20containers-storage%3A%24%7Buri%7D%0A%20%20if%20%5B%20%24%3F%20-eq%200%20%5D%3B%20then%20rm%20-rf%20%24%7Btar%7D%3B%20current_copy%3D%24%28%28current_copy%20%2B%201%29%29%3B%20fi%0Adone%20%3C%20%24%7BBINARY_FOLDER%7D%2F%24%7BOCP_RELEASE_LIST%7D%0A%0Aexit%200"
}
}
]
}
}
nodeNetwork:
config:
interfaces:
- name: ens1f0
type: ethernet
state: up
macAddress: "AA:BB:CC:11:22:33"
ipv4:
enabled: true
dhcp: true
ipv6:
enabled: false
interfaces:
- name: "ens1f0"
macAddress: "AA:BB:CC:11:22:33"
1 | Specifies the cluster image set used for deployment, unless you specify a different image set in the spec.clusters.clusterImageSetNameRef field. |
2 | Specifies the cluster image set used to deploy an individual cluster. If defined, it overrides the spec.clusterImageSetNameRef at the site level. |
The clusters.ignitionConfigOverride
field adds a configuration in Ignition format during the GitOps ZTP discovery stage.
The configuration includes systemd
services in the ISO mounted in virtual media. This way, the scripts are part of the discovery FCOS live ISO and they can be used to load the Assisted Installer (AI) images.
systemd
servicesThe systemd
services are var-mnt.mount
and precache-images.services
. The precache-images.service
depends on the disk partition to be mounted in /var/mnt
by the var-mnt.mount
unit.
The service calls a script called extract-ai.sh
.
extract-ai.sh
The extract-ai.sh
script extracts and loads the required images from the disk partition to the local container storage.
When the script finishes successfully, you can use the images locally.
agent-fix-bz1964591
The agent-fix-bz1964591
script is a workaround for an AI issue.
To prevent AI from removing the images, which can force the agent.service
to pull the images again from the registry, the agent-fix-bz1964591
script checks if the requested container images exist.
The nodes.installerArgs
field allows you to configure how the coreos-installer
utility writes the FCOS live ISO to disk. You need to indicate to save the disk partition labeled as data
because the artifacts saved in the data
partition are needed during the OKD installation stage.
The extra parameters are passed directly to the coreos-installer
utility that writes the live FCOS to disk.
On the next reboot, the operating system starts from the disk.
You can pass several options to the coreos-installer
utility:
OPTIONS:
...
-u, --image-url <URL>
Manually specify the image URL
-f, --image-file <path>
Manually specify a local image file
-i, --ignition-file <path>
Embed an Ignition config from a file
-I, --ignition-url <URL>
Embed an Ignition config from a URL
...
--save-partlabel <lx>...
Save partitions with this label glob
--save-partindex <id>...
Save partitions with this number or range
...
--insecure-ignition
Allow Ignition URL without HTTPS or hash
Similarly to clusters.ignitionConfigOverride
, the nodes.ignitionConfigOverride
field allows the addition of configurations in Ignition format to the coreos-installer
utility, but at the OKD installation stage.
When the FCOS is written to disk, the extra configuration included in the GitOps ZTP discovery ISO is no longer available. During the discovery stage, the extra configuration is stored in the memory of the live OS.
At this stage, the number of container images extracted and loaded is bigger than in the discovery stage. Depending on the OKD release and whether you install the Day-2 Operators, the installation time can vary. |
At the installation stage, the var-mnt.mount
and precache-ocp.services
systemd
services are used.
precache-ocp.service
The precache-ocp.service
depends on the disk partition to be mounted in /var/mnt
by the var-mnt.mount
unit.
The precache-ocp.service
service calls a script called extract-ocp.sh
.
To extract all the images before the OKD installation, you must execute |
extract-ocp.sh
The extract-ocp.sh
script extracts and loads the required images from the disk partition to the local container storage.
When you commit the SiteConfig
and optional PolicyGenerator
or PolicyGenTemplate
custom resources (CRs) to the Git repo that Argo CD is monitoring, you can start the GitOps ZTP workflow by syncing the CRs with the hub cluster.
When you download images by using a local or disconnected registry, you might see the The rendered catalog is invalid
error. This means that you are missing certificates of the new registry you want to pull content from.
The factory-precaching-cli tool image is built on a UBI Fedora image. Certificate paths and locations are the same on FCOS. |
Generating list of pre-cached artifacts...
error: unable to run command oc-mirror -c /mnt/imageset.yaml file:///tmp/fp-cli-3218002584/mirror --ignore-history --dry-run: Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/publish
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/v2
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/charts
Creating directory: /tmp/fp-cli-3218002584/mirror/oc-mirror-workspace/src/release-signatures
backend is not configured in /mnt/imageset.yaml, using stateless mode
backend is not configured in /mnt/imageset.yaml, using stateless mode
No metadata detected, creating new workspace
level=info msg=trying next host error=failed to do request: Head "https://eko4.cloud.lab.eng.bos.redhat.com:8443/v2/redhat/redhat-operator-index/manifests/v4.11": x509: certificate signed by unknown authority host=eko4.cloud.lab.eng.bos.redhat.com:8443
The rendered catalog is invalid.
Run "oc-mirror list operators --catalog CATALOG-NAME --package PACKAGE-NAME" for more information.
error: error rendering new refs: render reference "eko4.cloud.lab.eng.bos.redhat.com:8443/redhat/redhat-operator-index:v4.11": error resolving name : failed to do request: Head "https://eko4.cloud.lab.eng.bos.redhat.com:8443/v2/redhat/redhat-operator-index/manifests/v4.11": x509: certificate signed by unknown authority
Copy the registry certificate into your server:
# cp /tmp/eko4-ca.crt /etc/pki/ca-trust/source/anchors/.
Update the certificates truststore:
# update-ca-trust
Mount the host /etc/pki
folder into the factory-cli image:
# podman run -v /mnt:/mnt -v /root/.docker:/root/.docker -v /etc/pki:/etc/pki --privileged -it --rm quay.io/openshift-kni/telco-ran-tools:latest -- \
factory-precaching-cli download -r 4.17.0 --acm-version 2.5.4 \
--mce-version 2.0.4 -f /mnt \--img quay.io/custom/repository
--du-profile -s --skip-imageset