1. Documentation
×

etcd certificates

Purpose

etcd certificates are signed by the etcd-signer; they come from a certificate authority (CA) that is generated by the bootstrap process.

Expiration

The CA certificates are valid for 10 years. The peer, client, and server certificates are valid for three years.

Rotating the etcd certificate

Rotate the etcd certificate before it expires.

Procedure

  1. Make a backup copy of the current signer certificate by running the following command:

    $ oc get secret -n openshift-etcd etcd-signer -oyaml > signer_backup_secret.yaml

  2. Verify the remaining lifetime of the new signer certificate by running the following command:

    $ oc get secret -n openshift-etcd etcd-signer -ojsonpath='{.metadata.annotations.auth\.openshift\.io/certificate-not-after}'

  3. If the remaining lifetime is close to the current date, re-create the signer by deleting the signer and wait for the static pod roll out.

    • Delete the signer by running the following command:

      $ oc delete secret -n openshift-etcd etcd-signer

    • Wait for the static pod roll out by running the following command:

      $ oc wait --for=condition=Progressing=False --timeout=15m clusteroperator/etcd

  4. After etcd restarts, switch the original CA in the openshift-config namespace with the new, rotated one in openshift-etcd by running the following command:

    $ oc get secret etcd-signer -n openshift-etcd -ojson | jq 'del(.metadata["namespace","creationTimestamp","resourceVersion","selfLink","uid"])' | oc apply -n openshift-config -f -

  5. Wait for the cluster Operators to roll out and stabilize by running the following command:

    $ oc adm wait-for-stable-cluster --minimum-stable-period 2m

etcd certificate rotation alerts and metrics signer certificates

Two alert types inform users about pending etcd certificate expiration:

etcdSignerCAExpirationWarning

Occurs 730 days until the signer expires.

etcdSignerCAExpirationCritical

Occurs 365 days until the signer expires.

You can rotate the certificate for the following reasons:

  • You receive an expiration alert.

  • The private key is leaked.

When a private key is leaked, you must rotate all of the certificates.

There is an etcd signer for the OKD metrics system. Substitute the following metrics parameters in Rotating the etcd certificate.

  • etcd-metric-signer instead of etcd-signer

  • etcd-metrics-ca-bundle instead of etcd-ca-bundle

Management

These certificates are only managed by the system and are automatically rotated.

Services

etcd certificates are used for encrypted communication between etcd member peers and encrypted client traffic. The following certificates are generated and used by etcd and other processes that communicate with etcd:

  • Peer certificates: Used for communication between etcd members.

  • Client certificates: Used for encrypted server-client communication. Client certificates are currently used by the API server only, and no other service should connect to etcd directly except for the proxy. Client secrets (etcd-client, etcd-metric-client, etcd-metric-signer, and etcd-signer) are added to the openshift-config, openshift-monitoring, and openshift-kube-apiserver namespaces.

  • Server certificates: Used by the etcd server for authenticating client requests.

  • Metric certificates: All metric consumers connect to proxy with metric-client certificates.

Additional resources