1. Documentation
×

Using Operator Lifecycle Manager on restricted networks

For OKD clusters that are installed on restricted networks, also known as disconnected clusters, Operator Lifecycle Manager (OLM) by default cannot access the Red Hat-provided OperatorHub sources hosted on remote registries because those remote sources require full internet connectivity.

However, as a cluster administrator you can still enable your cluster to use OLM in a restricted network if you have a workstation that has full internet access. The workstation, which requires full internet access to pull the remote OperatorHub content, is used to prepare local mirrors of the remote sources, and push the content to a mirror registry.

The mirror registry can be located on a bastion host, which requires connectivity to both your workstation and the disconnected cluster, or a completely disconnected, or airgapped, host, which requires removable media to physically move the mirrored content to the disconnected environment.

This guide describes the following process that is required to enable OLM in restricted networks:

  • Disable the default remote OperatorHub sources for OLM.

  • Use a workstation with full internet access to create and push local mirrors of the OperatorHub content to a mirror registry.

  • Configure OLM to install and manage Operators from local sources on the mirror registry instead of the default remote sources.

After enabling OLM in a restricted network, you can continue to use your unrestricted workstation to keep your local OperatorHub sources updated as newer versions of Operators are released.

While OLM can manage Operators from local sources, the ability for a given Operator to run successfully in a restricted network still depends on the Operator itself meeting the following criteria:

  • List any related images, or other container images that the Operator might require to perform their functions, in the relatedImages parameter of its ClusterServiceVersion (CSV) object.

  • Reference all specified images by a digest (SHA) and not by a tag.

You can search software on the Red Hat Ecosystem Catalog for a list of Red Hat Operators that support running in disconnected mode by filtering with the following selections:

Type

Containerized application
Deployment method

Operator
Infrastructure features

Disconnected
Additional resources

Prerequisites

  • Log in to your OKD cluster as a user with cluster-admin privileges.

  • If you want to prune the default catalog and selectively mirror only a subset of Operators, install the opm CLI.

If you are using OLM in a restricted network on IBM Z, you must have at least 12 GB allocated to the directory where you place your registry.

Disabling the default OperatorHub sources

Operator catalogs that source content provided by Red Hat and community projects are configured for OperatorHub by default during an OKD installation. In a restricted network environment, you must disable the default catalogs as a cluster administrator. You can then configure OperatorHub to use local catalog sources.

Procedure

  • Disable the sources for the default catalogs by adding disableAllDefaultSources: true to the OperatorHub object:

    $ oc patch OperatorHub cluster --type json \
    -p '[{"op": "add", "path": "/spec/disableAllDefaultSources", "value": true}]'

Alternatively, you can use the web console to manage catalog sources. From the Administration → Cluster Settings → Configuration → OperatorHub page, click the Sources tab, where you can create, delete, disable, and enable individual sources.

Filtering a SQLite-based index image

An index image, based on the Operator bundle format, is a containerized snapshot of an Operator catalog. You can filter, or prune, an index of all but a specified list of packages, which creates a copy of the source index containing only the Operators that you want.

When configuring Operator Lifecycle Manager (OLM) to use mirrored content on restricted network OKD clusters, use this pruning method if you want to only mirror a subset of Operators from the default catalogs.

For the steps in this procedure, the target registry is an existing mirror registry that is accessible by your workstation with unrestricted network access. This example also shows pruning the index image for the default catalog catalog, but the process is the same for any index image.

Prerequisites

  • Workstation with unrestricted network access

  • podman version 1.9.3+

  • grpcurl (third-party command-line tool)

  • opm

  • Access to a registry that supports Docker v2-2

Procedure

  1. Authenticate with your target registry:

    $ podman login <target_registry>

  2. Determine the list of packages you want to include in your pruned index.

    1. Run the source index image that you want to prune in a container. For example:

      $ podman run -p50051:50051 \
    -it quay.io/operatorhubio/catalog:latest
      Example output
      Trying to pull quay.io/operatorhubio/catalog:latest...
Getting image source signatures
Copying blob ae8a0c23f5b1 done
...
INFO[0000] serving registry                              database=/database/index.db port=50051

    2. In a separate terminal session, use the grpcurl command to get a list of the packages provided by the index:

      $ grpcurl -plaintext localhost:50051 api.Registry/ListPackages > packages.out

    3. Inspect the packages.out file and identify which package names from this list you want to keep in your pruned index. For example:

      Example snippets of packages list
      ...
{
  "name": "couchdb-operator"
}
...
{
  "name": "eclipse-che"
}
...
{
{
  "name": "etcd"
}
...

    4. In the terminal session where you executed the podman run command, press Ctrl and C to stop the container process.

  3. Run the following command to prune the source index of all but the specified packages:

    $ opm index prune \
    -f quay.io/operatorhubio/catalog:latest \(1)
    -p couchdb-operator,eclipse-che,etcd \(2)
    [-i quay.io/openshift/origin-operator-registry:4.9.0] \(3)
    -t <target_registry>:<port>/<namespace>/catalog:latest (4)
    1 Index to prune.
    2 Comma-separated list of packages to keep.
    3 Required only for IBM Power and IBM Z images: Operator Registry base image with the tag that matches the target OKD cluster major and minor version.
    4 Custom tag for new index image being built.

  4. Run the following command to push the new index image to your target registry:

    $ podman push <target_registry>:<port>/<namespace>/catalog:latest

    where <namespace> is any existing namespace on the registry. For example, you might create an olm-mirror namespace to push all mirrored content to.

Mirroring an Operator catalog

For instructions about mirroring Operator catalogs for use with disconnected clusters, see Installing → Mirroring images for a disconnected installation.

Adding a catalog source to a cluster

Adding a catalog source to an OKD cluster enables the discovery and installation of Operators for users. Cluster administrators can create a CatalogSource object that references an index image. OperatorHub uses catalog sources to populate the user interface.

Prerequisites

  • An index image built and pushed to a registry.

Procedure

  1. Create a CatalogSource object that references your index image. If you used the oc adm catalog mirror command to mirror your catalog to a target registry, you can use the generated catalogSource.yaml file in your manifests directory as a starting point.

    1. Modify the following to your specifications and save it as a catalogSource.yaml file:

      apiVersion: operators.coreos.com/v1alpha1
kind: CatalogSource
metadata:
  name: my-operator-catalog (1)
  namespace: olm (2)
spec:
  sourceType: grpc
  image: <registry>/<namespace>/catalog:latest (3)
  displayName: My Operator Catalog
  publisher: <publisher_name> (4)
  updateStrategy:
    registryPoll: (5)
      interval: 30m
      1 If you mirrored content to local files before uploading to a registry, remove any backslash (/) characters from the metadata.name field to avoid an "invalid resource name" error when you create the object.
      2 If you want the catalog source to be available globally to users in all namespaces, specify the olm namespace. Otherwise, you can specify a different namespace for the catalog to be scoped and available only for that namespace.
      3 Specify your index image. If you specify a tag after the image name, for example :latest, the catalog source pod uses an image pull policy of Always, meaning the pod always pulls the image prior to starting the container. If you specify a digest, for example @sha256:<id>, the image pull policy is IfNotPresent, meaning the pod pulls the image only if it does not already exist on the node.
      4 Specify your name or an organization name publishing the catalog.
      5 Catalog sources can automatically check for new versions to keep up to date.

    2. Use the file to create the CatalogSource object:

      $ oc apply -f catalogSource.yaml

  2. Verify the following resources are created successfully.

    1. Check the pods:

      $ oc get pods -n olm
      Example output
      NAME                                    READY   STATUS    RESTARTS  AGE
my-operator-catalog-6njx6               1/1     Running   0         28s
marketplace-operator-d9f549946-96sgr    1/1     Running   0         26h

    2. Check the catalog source:

      $ oc get catalogsource -n olm
      Example output
      NAME                  DISPLAY               TYPE PUBLISHER  AGE
my-operator-catalog   My Operator Catalog   grpc            5s

    3. Check the package manifest:

      $ oc get packagemanifest -n olm
      Example output
      NAME                          CATALOG               AGE
jaeger-product                My Operator Catalog   93s

You can now install the Operators from the OperatorHub page on your OKD web console.

Additional resources

Updating a SQLite-based index image

After configuring OperatorHub to use a catalog source that references a custom index image, cluster administrators can keep the available Operators on their cluster up to date by adding bundle images to the index image.

You can update an existing index image using the opm index add command. For restricted networks, the updated content must also be mirrored again to the cluster.

Prerequisites

  • opm

  • podman version 1.9.3+

  • An index image built and pushed to a registry.

  • An existing catalog source referencing the index image.

Procedure

  1. Update the existing index by adding bundle images:

    $ opm index add \
    --bundles <registry>/<namespace>/<new_bundle_image>@sha256:<digest> \(1)
    --from-index <registry>/<namespace>/<existing_index_image>:<existing_tag> \(2)
    --tag <registry>/<namespace>/<existing_index_image>:<updated_tag> \(3)
    --pull-tool podman (4)
    1 The --bundles flag specifies a comma-separated list of additional bundle images to add to the index.
    2 The --from-index flag specifies the previously pushed index.
    3 The --tag flag specifies the image tag to apply to the updated index image.
    4 The --pull-tool flag specifies the tool used to pull container images.

    where:

    <registry>

    Specifies the hostname of the registry, such as quay.io or mirror.example.com.

    <namespace>

    Specifies the namespace of the registry, such as ocs-dev or abc.

    <new_bundle_image>

    Specifies the new bundle image to add to the registry, such as ocs-operator.

    <digest>

    Specifies the SHA image ID, or digest, of the bundle image, such as c7f11097a628f092d8bad148406aa0e0951094a03445fd4bc0775431ef683a41.

    <existing_index_image>

    Specifies the previously pushed image, such as abc-redhat-operator-index.

    <existing_tag>

    Specifies a previously pushed image tag, such as 4.10.

    <updated_tag>

    Specifies the image tag to apply to the updated index image, such as 4.10.1.

    Example command
    $ opm index add \
    --bundles quay.io/ocs-dev/ocs-operator@sha256:c7f11097a628f092d8bad148406aa0e0951094a03445fd4bc0775431ef683a41 \
    --from-index mirror.example.com/abc/abc-redhat-operator-index:4.10 \
    --tag mirror.example.com/abc/abc-redhat-operator-index:4.10.1 \
    --pull-tool podman

  2. Push the updated index image:

    $ podman push <registry>/<namespace>/<existing_index_image>:<updated_tag>

  3. Follow the steps in the Mirroring an Operator catalog procedure again to mirror the updated content. However, when you get to the step about creating the ImageContentSourcePolicy (ICSP) object, use the oc replace command instead of the oc create command. For example:

    $ oc replace -f ./manifests-catalog-<random_number>/imageContentSourcePolicy.yaml

    This change is required because the object already exists and must be updated.

    Normally, the oc apply command can be used to update existing objects that were previously created using oc apply. However, due to a known issue regarding the size of the metadata.annotations field in ICSP objects, the oc replace command must be used for this step currently.

  4. After Operator Lifecycle Manager (OLM) automatically polls the index image referenced in the catalog source at its regular interval, verify that the new packages are successfully added:

    $ oc get packagemanifests -n openshift-marketplace
Additional resources