openshift-network-diagnostics network-check-target-lpkp2 0/1 Running
Use the release notes to learn what is new or changed in the NVIDIA DPF Operator.
The NVIDIA DPF Operator on OKD has known limitations for uninstall, secrets, MTU, secure boot, multi-DPU hosts, and HBN, unsupported OVN-Kubernetes features, and issues that can affect Grafana and DTS metrics.
New features and enhancements
Added comprehensive DPU telemetry monitoring through the DOCA Telemetry Service (DTS) with built-in OKD Console dashboard integration. DTS metrics are now accessible directly through the OKD web console without requiring additional tools.
The DPF HCP Provisioner Operator provides enhanced lifecycle management for DPU hosted clusters, including automatic CSR approval, kubeconfig injection, and BlueField container image lookup.
A comprehensive traffic validation framework with pre-configured test pods uses nicolaka/netshoot containers to validate end-to-end DPU service chain functionality.
Expanded diagnostic tools and troubleshooting procedures cover DPU provisioning, hosted cluster management, networking issues, and comprehensive log collection.
Bug fixes
Fixed issues with BlueField Bootstream File (BFB) image download and verification processes.
Resolved Node Feature Discovery (NFD) compatibility issues for reliable DPU hardware detection.
Fixed OVN-Kubernetes integration issues that could cause worker nodes to remain in NotReady state.
Technology Preview features
|
The NVIDIA DPF Operator is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope. |
Known issues and limitations
Only x86_64 worker nodes are supported in this release. ARM-based DPU workers are not supported.
The DPF Operator does not support an automated uninstall. If you must remove DPF, set spec.manageDPUServiceTemplates to false in the DPFHCPProvisionerConfig resource before you uninstall. This prevents the DPF HCP Provisioner Operator from continuing to manage DPUServiceTemplate resources during the uninstall process.
The pull secret and SSH secret references are immutable after creation and cannot be modified. Ensure that each secret contains the correct data before you create it and reference it in the DPFHCPProvisioner custom resource.
Secondary pod interfaces (MultiNetwork) are not supported.
You cannot change the MTU value after deployment.
To boot the RHCOS BFB image with secure boot enabled, the DPU firmware must be at version 3.1.0 or later. Use a BFB firmware bundle to upgrade the firmware.
Hosts with more than one DPU are not supported.
Redeploying a DPUDeployment is not supported in this release.
Because of a limitation in the resource injector, a deployment can target either the DPU workers or all other nodes, but not both.
An HBN daemon set pod might remain in the FailedCreatePodSandBox state. As a workaround, delete and re-create the affected pods. For more information, see OCPBUGS-100251.
Changing the ocpReleaseImage of a HostedCluster while an upgrade is already in progress is not supported.
When a DPU reboots, the corresponding DPU worker node loses connectivity and a NoExecute taint is added to the host. Most pods are evicted immediately, but some daemon set pods remain and might lose connectivity until you re-create them. For example:
openshift-network-diagnostics network-check-target-lpkp2 0/1 Running
The NVIDIA Maintenance Operator might fail to pause the machine config pool, which leaves the DPU in the NodeEffect or Initializing state. As a workaround, pause the worker-dpu machine config pool manually:
$ oc patch mcp worker-dpu --type merge -p '{
"spec": {"paused": true},
"metadata": {"annotations": {"maintenance.nvidia.com/mcp-paused": "true"}}
}'
After an IPMI reset reboot, workload pods might fail to recover because of a known kubelet bug (Kubernetes issue 128043) that prevents virtual function (VF) devices from being re-created immediately at startup. This does not break functionality, but it leaves the cluster in an inconsistent state. Standard and IPMI2 reboots recover cleanly. As a workaround, re-create the affected pods manually if needed.
After a node reboot or DPU redeployment, the SR-IOV device plugin might publish the node’s virtual function (VF) resource count before all VFs are created. The init container unblocks when the first VF appears instead of waiting for all configured VFs, so the reported openshift.io/bf3_vfs capacity can be lower than expected. As a workaround, restart the SR-IOV device plugin pod on the affected node, after which the full count is reported.
OVN-Kubernetes feature support
The following table lists the support and hardware offload status of OVN-Kubernetes features in this Technology Preview.
| Feature | Supported | Offloaded |
|---|---|---|
Administrative Network Policies (ANP) |
Yes |
Yes |
Egress IP |
Yes |
No |
Egress Firewall |
Yes |
No |
Egress Quality of Service (QoS) |
Yes |
No |
Secondary networks |
No |
No |
User Defined Networks (UDN) |
No |
No |
Quality of Service (QoS) |
No |
No |
Multiple External Gateways (MEG) |
No |
No |
OVN-Kubernetes identity |
No |
No |
Border Gateway Protocol (BGP) |
No |
No |
Multicast |
No |
No |
Hybrid Overlay |
No |
No |
Local gateway mode |
No |
No |
IPFIX or NetFlow sampling |
No |
No |
Grafana deployment issues
Grafana pods might be scheduled on worker nodes that depend on DPU networking, which creates a circular dependency.
Configure Grafana to run on control plane nodes by adding nodeSelector and tolerations to the Grafana custom resource:
spec:
deployment:
spec:
template:
spec:
nodeSelector:
node-role.kubernetes.io/control-plane: ""
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
DTS metrics collection issues
The ServiceMonitor might not be configured correctly, or DTS pods might not be running.
Verify that DTS pods are running:
$ oc get pods -n dpf-operator-system -l app=dts
Check the ServiceMonitor configuration:
$ oc get servicemonitor -n dpf-operator-system
$ oc describe servicemonitor <servicemonitor-name> -n dpf-operator-system
Verify that user workload monitoring is enabled:
$ oc get configmap cluster-monitoring-config -n openshift-monitoring -o yaml
Check Prometheus targets to ensure that DTS endpoints are being scraped. Access the Prometheus web console and go to Status → Targets to verify that DTS endpoints are listed and healthy.