1. Documentation
×

Using the must-gather tool

You can collect logs, metrics, and information about OADP custom resources by using the must-gather tool. The must-gather data must be attached to all customer cases.

You can run the must-gather tool with the following data collection options:

  • Full must-gather data collection collects Prometheus metrics, pod logs, and Velero CR information for all namespaces where the OADP Operator is installed.

  • Essential must-gather data collection collects pod logs and Velero CR information for a specific duration of time, for example, one hour or 24 hours. Prometheus metrics and duplicate logs are not included.

  • must-gather data collection with timeout. Data collection can take a long time if there are many failed Backup CRs. You can improve performance by setting a timeout value.

  • Prometheus metrics data dump downloads an archive file containing the metrics data collected by Prometheus.

Prerequisites

  • You have logged in to the OKD cluster as a user with the cluster-admin role.

  • You have installed the OpenShift CLI (oc).

  • You must use Fedora 9 with OADP 1.4.

Procedure

  1. Navigate to the directory where you want to store the must-gather data.

  2. Run the oc adm must-gather command for one of the following data collection options:

    • For full must-gather data collection, including Prometheus metrics, run the following command:

      $ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4

      The data is saved as must-gather/must-gather.tar.gz. You can upload this file to a support case on the Red Hat Customer Portal.

      For essential must-gather data collection, without Prometheus metrics, for a specific time duration, run the following command:

      $ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 \
  -- /usr/bin/gather_<time>_essential (1)
      1 Specify the time in hours. Allowed values are 1h, 6h, 24h, 72h, or all, for example, gather_1h_essential or gather_all_essential.

    • For must-gather data collection with timeout, run the following command:

      $ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 \
  -- /usr/bin/gather_with_timeout <timeout> (1)
      1 Specify a timeout value in seconds.

    • For a Prometheus metrics data dump, run the following command:

      $ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 -- /usr/bin/gather_metrics_dump

      This operation can take a long time. The data is saved as must-gather/metrics/prom_data.tar.gz.

Additional resources

Using must-gather with insecure TLS connections

If a custom CA certificate is used, the must-gather pod fails to grab the output for velero logs/describe. To use the must-gather tool with insecure TLS connections, you can pass the gather_without_tls flag to the must-gather command.

Procedure

  • Pass the gather_without_tls flag, with value set to true, to the must-gather tool by using the following command:

$ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 -- /usr/bin/gather_without_tls <true/false>

By default, the flag value is set to false. Set the value to true to allow insecure TLS connections.

Combining options when using the must-gather tool

Currently, it is not possible to combine must-gather scripts, for example specifying a timeout threshold while permitting insecure TLS connections. In some situations, you can get around this limitation by setting up internal variables on the must-gather command line, such as the following example:

$ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 -- skip_tls=true /usr/bin/gather_with_timeout <timeout_value_in_seconds>

In this example, set the skip_tls variable before running the gather_with_timeout script. The result is a combination of gather_with_timeout and gather_without_tls.

The only other variables that you can specify this way are the following:

  • logs_since, with a default value of 72h

  • request_timeout, with a default value of 0s

If DataProtectionApplication custom resource (CR) is configured with s3Url and insecureSkipTLS: true, the CR does not collect the necessary logs because of a missing CA certificate. To collect those logs, run the must-gather command with the following option:

$ oc adm must-gather --image=registry.redhat.io/oadp/oadp-mustgather-rhel9:v1.4 -- /usr/bin/gather_without_tls true