1. Documentation
×

Configuring a custom PKI

To ensure secure communication between internal components, your OKD cluster uses a shared set of trusted Certificate Authorities (CAs). If your organization uses its own private certificates (a custom PKI), you must add your CA to the cluster so that all components trust it.

You can add your custom CA certificates to the cluster-wide truststore in one of two ways:

  • During cluster installation, by adding your CA certificate to the install-config.yaml file.

  • On a running cluster, by creating a ConfigMap object that contains your CA certificate and referencing it in the cluster Proxy object.

The cluster Proxy object is the mechanism for managing the cluster-wide truststore. This guide focuses only on the task of adding a CA. If you also need to configure an egress proxy, refer to the "Configuring the cluster-wide proxy" chapter for detailed instructions.

Adding a custom CA during cluster installation

You can add a custom CA to the cluster-wide truststore during installation by providing the certificate in your install-config.yaml file.

This procedure uses the additionalTrustBundle parameter. If you are also configuring an egress proxy, you can add this parameter to your install-config.yaml file along with your proxy configuration. For more information on the available proxy settings, see the "Configuring the cluster-wide proxy" chapter.

Prerequisites

  • You have access to the install-config.yaml file for your cluster installation.

  • You have your custom CA certificate avalable in PEM-encoded format.

Procedure

  1. Open your install-config.yaml file.

  2. Add the additionalTrustBundle parameter with your PEM-encoded CA certificate:

    apiVersion: v1
baseDomain: my.domain.com
metadata:
  name: my-cluster
additionalTrustBundle: | (1)
  -----BEGIN CERTIFICATE-----
  <MY_PEM_ENCODED_CA_CERT>
  -----END CERTIFICATE-----
    1 The additionalTrustBundle parameter contains the custom CA certificate that you want the cluster to trust. The installation program uses the certificate to generate a user-ca-bundle ConfigMap object in the openshift-config namespace.

  3. Save the install-config.yaml file and continue with your cluster installation.

During installation, the Cluster Network Operator (CNO) merges the certificate you provided with the system’s default trust bundle. This process makes your custom CA trusted across the entire cluster.

Adding a custom CA to a running cluster

For a running cluster, you can add a custom CA by creating a ConfigMap object that contains your certificate and then referencing that ConfigMap object in the cluster Proxy object.

When you modify the cluster Proxy object, the Machine Config Operator (MCO) initiates a rolling reboot of all nodes to apply the change. This is expected behavior and does not require manual intervention.

This procedure uses the trustedCA field in the Proxy object. If you also need to configure or modify egress proxy settings at the same time, see the "Configuring the cluster-wide proxy" chapter for detailed instructions.

Prerequisites

  • You have cluster-admin privileges.

  • You have the OpenShift CLI (oc) installed.

  • You have your custom CA certificate available in PEM-encoded format.

Procedure

The procedure involves two stages: creating a ConfigMap object with your certificate and then updating the cluster to trust it.

  1. Create a ConfigMap object with your CA certificate.

    1. Create a YAML file named custom-ca.yaml to define the ConfigMap object.

    2. Add the following content to the file:

      apiVersion: v1
kind: ConfigMap
metadata:
  name: custom-ca-bundle      (1)
  namespace: openshift-config  (2)
data:
  ca-bundle.crt: |         (3)
    -----BEGIN CERTIFICATE-----
    <MY_PEM_ENCODED_CA_CERT>
    -----END CERTIFICATE-----
      1 The name of the ConfigMap object that you will reference from the Proxy object.
      2 The ConfigMap object must be created in the openshift-config namespace.
      3 The data key for the certificate bundle must be ca-bundle.crt.

  2. Apply the manifest to create the ConfigMap object in the cluster:

    $ oc apply -f custom-ca.yaml

  3. Reference the ConfigMap object in the cluster Proxy object.

    1. Run the following oc patch command to update the cluster Proxy object to reference the ConfigMap object you just created.

      $ oc patch proxy/cluster --type=merge --patch='{"spec":{"trustedCA":{"name":"custom-ca-bundle"}}}'

After you run this command, the Machine Config Operator (MCO) detects the change and begins distributing the new trusted CA to all nodes in the cluster.

Verifying the custom CA configuration

After you add your custom CA certificate, you can verify that it has been successfully added to the cluster-wide trust bundle.

Prerequisites

  • You have permissions to view ConfigMap objects in the openshift-config namespace.

  • You have the OpenShift CLI (oc) installed.

Procedure

  1. Run the following command to view the contents of the cluster-wide CA trust bundle:

    $ oc get configmap trusted-ca-bundle -n openshift-config -o yaml

  2. In the YAML output, inspect the data.ca-bundle.crt field. This field contains all the trusted certificates for the cluster.

  3. Verify that the PEM-encoded certificate you added is included in the list of certificates. The output will resemble the following structure:

    kind: ConfigMap
metadata:
  name: trusted-ca-bundle
  namespace: openshift-config
data:
  ca-bundle.crt: |
    -----BEGIN CERTIFICATE-----
    <A_SYSTEM_CA_CERTIFICATE>
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    <ANOTHER_SYSTEM_CA_CERTIFICATE>
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    <YOUR_CUSTOM_CA_CERTIFICATE_SHOULD_BE_HERE>
    -----END CERTIFICATE-----

    If your certificate is present in the output, the cluster now trusts your custom PKI.

Certificate injection using Operators

Once your custom CA certificate is added to the cluster via ConfigMap, the Cluster Network Operator merges the user-provided and system CA certificates into a single bundle and injects the merged bundle into the Operator requesting the trust bundle injection.

After adding a config.openshift.io/inject-trusted-cabundle="true" label to the config map, existing data in it is deleted. The Cluster Network Operator takes ownership of a config map and only accepts ca-bundle as data. You must use a separate config map to store service-ca.crt by using the service.beta.openshift.io/inject-cabundle=true annotation or a similar configuration. Adding a config.openshift.io/inject-trusted-cabundle="true" label and service.beta.openshift.io/inject-cabundle=true annotation on the same config map can cause issues.

Operators request this injection by creating an empty ConfigMap with the following label:

config.openshift.io/inject-trusted-cabundle="true"

An example of the empty ConfigMap:

apiVersion: v1
data: {}
kind: ConfigMap
metadata:
  labels:
    config.openshift.io/inject-trusted-cabundle: "true"
  name: ca-inject (1)
  namespace: apache
1 Specifies the empty ConfigMap name.

The Operator mounts this ConfigMap into the container’s local trust store.

Adding a trusted CA certificate is only needed if the certificate is not included in the Fedora CoreOS (FCOS) trust bundle.

Certificate injection is not limited to Operators. The Cluster Network Operator injects certificates across any namespace when an empty ConfigMap is created with the config.openshift.io/inject-trusted-cabundle=true label.

The ConfigMap can reside in any namespace, but the ConfigMap must be mounted as a volume to each container within a pod that requires a custom CA. For example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-example-custom-ca-deployment
  namespace: my-example-custom-ca-ns
spec:
  ...
    spec:
      ...
      containers:
        - name: my-container-that-needs-custom-ca
          volumeMounts:
          - name: trusted-ca
            mountPath: /etc/pki/ca-trust/extracted/pem
            readOnly: true
      volumes:
      - name: trusted-ca
        configMap:
          name: ca-inject
          items:
            - key: ca-bundle.crt (1)
              path: tls-ca-bundle.pem (2)
1 ca-bundle.crt is required as the ConfigMap key.
2 tls-ca-bundle.pem is required as the ConfigMap path.