apiVersion: v1
kind: Secret
metadata:
namespace: <target_namespace> (1)
name: <target_secret_name> (2)
data:
service_account.json: <service_account> (3)
Manual mode with GCP Workload Identity is supported for Google Cloud Platform (GCP).
This credentials strategy is supported for only new OKD clusters and must be configured during installation. You cannot reconfigure an existing cluster that uses a different credentials strategy to use this feature. |
In manual mode with GCP Workload Identity, the individual OKD cluster components can impersonate IAM service accounts using short-term, limited-privilege credentials.
Requests for new and refreshed credentials are automated by using an appropriately configured OpenID Connect (OIDC) identity provider, combined with IAM service accounts. OKD signs service account tokens that are trusted by GCP, and can be projected into a pod and used for authentication. Tokens are refreshed after one hour by default.
Using manual mode with GCP Workload Identity changes the content of the GCP credentials that are provided to individual OKD components.
apiVersion: v1
kind: Secret
metadata:
namespace: <target_namespace> (1)
name: <target_secret_name> (2)
data:
service_account.json: <service_account> (3)
1 | The namespace for the component. |
2 | The name of the component secret. |
3 | The Base64 encoded service account. |
service_account.json
file using long-lived credentials{
"type": "service_account", (1)
"project_id": "<project_id>",
"private_key_id": "<private_key_id>",
"private_key": "<private_key>", (2)
"client_email": "<client_email_address>",
"client_id": "<client_id>",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/<client_email_address>"
}
1 | The credential type is service_account . |
2 | The private RSA key that is used to authenticate to GCP. This key must be kept secure and is not rotated. |
service_account.json
file using GCP Workload Identity{
"type": "external_account", (1)
"audience": "//iam.googleapis.com/projects/123456789/locations/global/workloadIdentityPools/test-pool/providers/test-provider", (2)
"subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
"token_url": "https://sts.googleapis.com/v1/token",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/<client_email_address>:generateAccessToken", (3)
"credential_source": {
"file": "<path_to_token>", (4)
"format": {
"type": "text"
}
}
}
1 | The credential type is external_account . |
2 | The target audience is the GCP Workload Identity provider. |
3 | The resource URL of the service account that can be impersonated with these credentials. |
4 | The path to the service account token inside the pod. By convention, this is /var/run/secrets/openshift/serviceaccount/token for OKD components. |
To install a cluster that is configured to use the Cloud Credential Operator (CCO) in manual mode with GCP Workload Identity:
To create and manage cloud credentials from outside of the cluster when the Cloud Credential Operator (CCO) is operating in
manual mode with GCP Workload Identity,
extract and prepare the CCO utility (ccoctl
) binary.
The |
Obtain the OKD release image by running the following command:
$ RELEASE_IMAGE=$(./openshift-install version | awk '/release image/ {print $3}')
Obtain the CCO container image from the OKD release image by running the following command:
$ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE -a ~/.pull-secret)
Ensure that the architecture of the |
Extract the ccoctl
binary from the CCO container image within the OKD release image by running the following command:
$ oc image extract $CCO_IMAGE --file="/usr/bin/ccoctl" -a ~/.pull-secret
Change the permissions to make ccoctl
executable by running the following command:
$ chmod 775 ccoctl
To verify that ccoctl
is ready to use, display the help file by running the following command:
$ ccoctl --help
ccoctl --help
:OpenShift credentials provisioning tool
Usage:
ccoctl [command]
Available Commands:
alibabacloud Manage credentials objects for alibaba cloud
aws Manage credentials objects for AWS cloud
gcp Manage credentials objects for Google cloud
help Help about any command
ibmcloud Manage credentials objects for IBM Cloud
nutanix Manage credentials objects for Nutanix
Flags:
-h, --help help for ccoctl
Use "ccoctl [command] --help" for more information about a command.
You can use the ccoctl gcp create-all
command to automate the creation of GCP resources.
By default, |
You must have:
Extracted and prepared the ccoctl
binary.
Extract the list of CredentialsRequest
objects from the OKD release image:
$ oc adm release extract \
--credentials-requests \
--cloud=gcp \
--to=<path_to_directory_with_list_of_credentials_requests>/credrequests \ (1)
quay.io/<path_to>/ocp-release:<version>
1 | credrequests is the directory where the list of CredentialsRequest objects is stored. This command creates the directory if it does not exist. |
This command can take a few moments to run. |
Use the ccoctl
tool to process all CredentialsRequest
objects in the credrequests
directory:
$ ccoctl gcp create-all \
--name=<name> \
--region=<gcp_region> \
--project=<gcp_project_id> \
--credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests
where:
<name>
is the user-defined name for all created GCP resources used for tracking.
<gcp_region>
is the GCP region in which cloud resources will be created.
<gcp_project_id>
is the GCP project ID in which cloud resources will be created.
<path_to_directory_with_list_of_credentials_requests>/credrequests
is the directory containing the files of CredentialsRequest
manifests to create GCP service accounts.
If your cluster uses Technology Preview features that are enabled by the |
To verify that the OKD secrets are created, list the files in the <path_to_ccoctl_output_dir>/manifests
directory:
$ ls <path_to_ccoctl_output_dir>/manifests
You can verify that the IAM service accounts are created by querying GCP. For more information, refer to GCP documentation on listing IAM service accounts.
Configure an account with the cloud platform that hosts your cluster.
Obtain the OKD release image.
Change to the directory that contains the installation program and create the install-config.yaml
file:
$ openshift-install create install-config --dir <installation_directory>
where <installation_directory>
is the directory in which the installation program creates files.
Edit the install-config.yaml
configuration file so that it contains the credentialsMode
parameter set to Manual
.
install-config.yaml
configuration fileapiVersion: v1
baseDomain: cluster1.example.com
credentialsMode: Manual (1)
compute:
- architecture: amd64
hyperthreading: Enabled
1 | This line is added to set the credentialsMode parameter to Manual . |
Create the required OKD installation manifests:
$ openshift-install create manifests
Copy the manifests that ccoctl
generated to the manifests directory that the installation program created:
$ cp /<path_to_ccoctl_output_dir>/manifests/* ./manifests/
Copy the private key that the ccoctl
generated in the tls
directory to the installation directory:
$ cp -a /<path_to_ccoctl_output_dir>/tls .
Run the OKD installer:
$ ./openshift-install create cluster
Connect to the OKD cluster.
Verify that the cluster does not have root
credentials:
$ oc get secrets -n kube-system gcp-credentials
The output should look similar to:
Error from server (NotFound): secrets "gcp-credentials" not found
Verify that the components are assuming the service accounts that are specified in the secret manifests, instead of using credentials that are created by the CCO:
$ oc get secrets -n openshift-image-registry installer-cloud-credentials -o json | jq -r '.data."service_account.json"' | base64 -d
The output should show the role and web identity token that are used by the component and look similar to:
{
"type": "external_account", (1)
"audience": "//iam.googleapis.com/projects/123456789/locations/global/workloadIdentityPools/test-pool/providers/test-provider",
"subject_token_type": "urn:ietf:params:oauth:token-type:jwt",
"token_url": "https://sts.googleapis.com/v1/token",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/<client-email-address>:generateAccessToken", (2)
"credential_source": {
"file": "/var/run/secrets/openshift/serviceaccount/token",
"format": {
"type": "text"
}
}
}
1 | The credential type is external_account . |
2 | The resource URL of the service account used by the Image Registry Operator. |
The release image for the version of OKD that you are upgrading to contains a version of the ccoctl
binary and list of CredentialsRequest
objects specific to that release.
To create and manage cloud credentials from outside of the cluster when the Cloud Credential Operator (CCO) is operating in
manual mode with GCP Workload Identity,
extract and prepare the CCO utility (ccoctl
) binary.
The |
Obtain the OKD release image by running the following command:
$ RELEASE_IMAGE=$(./openshift-install version | awk '/release image/ {print $3}')
Obtain the CCO container image from the OKD release image by running the following command:
$ CCO_IMAGE=$(oc adm release info --image-for='cloud-credential-operator' $RELEASE_IMAGE -a ~/.pull-secret)
Ensure that the architecture of the |
Extract the ccoctl
binary from the CCO container image within the OKD release image by running the following command:
$ oc image extract $CCO_IMAGE --file="/usr/bin/ccoctl" -a ~/.pull-secret
Change the permissions to make ccoctl
executable by running the following command:
$ chmod 775 ccoctl
To verify that ccoctl
is ready to use, display the help file by running the following command:
$ ccoctl --help
ccoctl --help
:OpenShift credentials provisioning tool
Usage:
ccoctl [command]
Available Commands:
alibabacloud Manage credentials objects for alibaba cloud
aws Manage credentials objects for AWS cloud
gcp Manage credentials objects for Google cloud
help Help about any command
ibmcloud Manage credentials objects for IBM Cloud
nutanix Manage credentials objects for Nutanix
Flags:
-h, --help help for ccoctl
Use "ccoctl [command] --help" for more information about a command.
The process for upgrading an OKD cluster configured for manual mode with GCP Workload Identity is similar to creating the cloud provider resources during installation.
By default, |
Obtain the OKD release image for the version that you are upgrading to.
Extract and prepare the ccoctl
binary from the release image.
Extract the list of CredentialsRequest
custom resources (CRs) from the OKD release image by running the following command:
$ oc adm release extract --credentials-requests \
--cloud=gcp \
--to=<path_to_directory_with_list_of_credentials_requests>/credrequests \ (1)
quay.io/<path_to>/ocp-release:<version>
1 | credrequests is the directory where the list of CredentialsRequest objects is stored. This command creates the directory if it does not exist. |
For each CredentialsRequest
CR in the release image, ensure that a namespace that matches the text in the spec.secretRef.namespace
field exists in the cluster. This field is where the generated secrets that hold the credentials configuration are stored.
CredentialsRequest
objectapiVersion: cloudcredential.openshift.io/v1
kind: CredentialsRequest
metadata:
annotations:
exclude.release.openshift.io/internal-openshift-hosted: "true"
include.release.openshift.io/self-managed-high-availability: "true"
name: cloud-credential-operator-gcp-ro-creds
namespace: openshift-cloud-credential-operator
spec:
providerSpec:
apiVersion: cloudcredential.openshift.io/v1
kind: GCPProviderSpec
predefinedRoles:
- roles/iam.securityReviewer
- roles/iam.roleViewer
skipServiceCheck: true
secretRef:
name: cloud-credential-operator-gcp-ro-creds
namespace: openshift-cloud-credential-operator (1)
serviceAccountNames:
- cloud-credential-operator
1 | This field indicates the namespace which needs to exist to hold the generated secret. |
For any CredentialsRequest
CR for which the cluster does not already have a namespace with the name specified in spec.secretRef.namespace
, create the namespace by running the following command:
$ oc create namespace <component_namespace>
Use the ccoctl
tool to process all CredentialsRequest
objects in the credrequests
directory by running the following command:
$ ccoctl gcp create-service-accounts \
--credentials-requests-dir=<path_to_directory_with_list_of_credentials_requests>/credrequests \
--name=<name> \
--project=<gcp_project_id> \
--workload-identity-pool=<name> \
--workload-identity-provider=<name>
where:
<path_to_directory_with_list_of_credentials_requests>/credrequests
is the directory containing the files of CredentialsRequest
manifests to create GCP service accounts.
<name>
is the user-defined name for all created GCP resources used for tracking.
<gcp_project_id>
is the GCP project ID in which cloud resources will be created.
If your cluster uses Technology Preview features that are enabled by the |
For each CredentialsRequest
object, ccoctl
creates
a service account
and a permissions policy as defined in each CredentialsRequest
object from the OKD release image.
Apply the secrets to your cluster by running the following command:
$ ls <path_to_ccoctl_output_dir>/manifests/*-credentials.yaml | xargs -I{} oc apply -f {}
You can verify that the service accounts are created by querying GCP. For more information, refer to GCP documentation on listing service accounts.
The Cloud Credential Operator (CCO) Upgradable
status for a cluster with manually maintained credentials is False
by default.
For minor releases, for example, from 4.10 to 4.11, this status prevents you from upgrading until you have addressed any updated permissions and annotated the CloudCredential
resource to indicate that the permissions are updated as needed for the next version. This annotation changes the Upgradable
status to True
.
For z-stream releases, for example, from 4.11.0 to 4.11.1, no permissions are added or changed, so the upgrade is not blocked.
Before upgrading a cluster with manually maintained credentials, you must create any new credentials for the release image that you are upgrading to. Additionally, you must review the required permissions for existing credentials and accommodate any new permissions requirements in the new release for those components.
Extract and examine the CredentialsRequest
custom resource for the new release.
The "Manually creating IAM" section of the installation content for your cloud provider explains how to obtain and use the credentials required for your cloud.
Update the manually maintained credentials on your cluster:
Create new secrets for any CredentialsRequest
custom resources that are added by the new release image.
If the CredentialsRequest
custom resources for any existing credentials that are stored in secrets have changed their permissions requirements, update the permissions as required.
When all of the secrets are correct for the new release, indicate that the cluster is ready to upgrade:
Log in to the OKD CLI as a user with the cluster-admin
role.
Edit the CloudCredential
resource to add an upgradeable-to
annotation within the metadata
field:
$ oc edit cloudcredential cluster
...
metadata:
annotations:
cloudcredential.openshift.io/upgradeable-to: <version_number>
...
Where <version_number>
is the version you are upgrading to, in the format x.y.z
. For example, 4.8.2
for OKD 4.8.2.
It may take several minutes after adding the annotation for the upgradeable status to change.
In the Administrator perspective of the web console, navigate to Administration → Cluster Settings.
To view the CCO status details, click cloud-credential in the Cluster Operators list.
If the Upgradeable status in the Conditions section is False, verify that the upgradeable-to
annotation is free of typographical errors.
When the Upgradeable status in the Conditions section is True, you can begin the OKD upgrade.