$ cat << EOF | oc apply -f -
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: openshift-nmstate
name: openshift-nmstate
name: openshift-nmstate
spec:
finalizers:
- kubernetes
EOF
The Kubernetes NMState Operator provides a Kubernetes API for performing state-driven network configuration across the OKD cluster’s nodes with NMState. The Kubernetes NMState Operator provides users with functionality to configure various network interface types, DNS, and routing on cluster nodes. Additionally, the daemons on the cluster nodes periodically report on the state of each node’s network interfaces to the API server.
Red Hat supports the Kubernetes NMState Operator in production environments on bare-metal, IBM Power®, IBM Z®, IBM® LinuxONE, VMware vSphere, and OpenStack installations. Red Hat support exists for using the Kubernetes NMState Operator on Microsoft Azure but in a limited capacity. Support is limited to configuring DNS servers on your system as a postinstallation task. |
Before you can use NMState with OKD, you must install the Kubernetes NMState Operator.
The Kubernetes NMState Operator updates the network configuration of a secondary NIC. The Operator cannot update the network configuration of the primary NIC, or update the On a bare-metal platform, using the Kubernetes NMState Operator to update the |
OKD uses nmstate
to report on and configure the state of the node network. This makes it possible to modify the network policy configuration, such as by creating a Linux bridge on all nodes, by applying a single configuration manifest to the cluster.
Node networking is monitored and updated by the following objects:
NodeNetworkState
Reports the state of the network on that node.
NodeNetworkConfigurationPolicy
Describes the requested network configuration on nodes. You update the node network configuration, including adding and removing interfaces, by applying a NodeNetworkConfigurationPolicy
CR to the cluster.
NodeNetworkConfigurationEnactment
Reports the network policies enacted upon each node.
You can install the Kubernetes NMState Operator by using the web console or the CLI.
You can install the Kubernetes NMState Operator by using the web console. After it is installed, the Operator can deploy the NMState State Controller as a daemon set across all of the cluster nodes.
You are logged in as a user with cluster-admin
privileges.
Select Operators → OperatorHub.
In the search field below All Items, enter nmstate
and click Enter to search for the Kubernetes NMState Operator.
Click on the Kubernetes NMState Operator search result.
Click on Install to open the Install Operator window.
Click Install to install the Operator.
After the Operator finishes installing, click View Operator.
Under Provided APIs, click Create Instance to open the dialog box for creating an instance of kubernetes-nmstate
.
In the Name field of the dialog box, ensure the name of the instance is nmstate.
The name restriction is a known issue. The instance is a singleton for the entire cluster. |
Accept the default settings and click Create to create the instance.
After you install the Kubernetes NMState Operator, the Operator has deployed the NMState State Controller as a daemon set across all of the cluster nodes.
You can install the Kubernetes NMState Operator by using the OpenShift CLI (oc)
. After it is installed, the Operator can deploy the NMState State Controller as a daemon set across all of the cluster nodes.
You have installed the OpenShift CLI (oc
).
You are logged in as a user with cluster-admin
privileges.
Create the nmstate
Operator namespace:
$ cat << EOF | oc apply -f -
apiVersion: v1
kind: Namespace
metadata:
labels:
kubernetes.io/metadata.name: openshift-nmstate
name: openshift-nmstate
name: openshift-nmstate
spec:
finalizers:
- kubernetes
EOF
Create the OperatorGroup
:
$ cat << EOF | oc apply -f -
apiVersion: operators.coreos.com/v1
kind: OperatorGroup
metadata:
annotations:
olm.providedAPIs: NMState.v1.nmstate.io
name: openshift-nmstate
namespace: openshift-nmstate
spec:
targetNamespaces:
- openshift-nmstate
EOF
Subscribe to the nmstate
Operator:
$ cat << EOF| oc apply -f -
apiVersion: operators.coreos.com/v1alpha1
kind: Subscription
metadata:
labels:
operators.coreos.com/kubernetes-nmstate-operator.openshift-nmstate: ""
name: kubernetes-nmstate-operator
namespace: openshift-nmstate
spec:
channel: stable
installPlanApproval: Automatic
name: kubernetes-nmstate-operator
source: redhat-operators
sourceNamespace: openshift-marketplace
EOF
Confirm the ClusterServiceVersion
(CSV) status for the nmstate
Operator deployment equals Succeeded
:
$ oc get clusterserviceversion -n openshift-nmstate \
-o custom-columns=Name:.metadata.name,Phase:.status.phase
Name Phase
kubernetes-nmstate-operator.4.17.0-202210210157 Succeeded
Create an instance of the nmstate
Operator:
$ cat << EOF | oc apply -f -
apiVersion: nmstate.io/v1
kind: NMState
metadata:
name: nmstate
EOF
Verify the pods for NMState Operator are running:
$ oc get pod -n openshift-nmstate
Name Ready Status Restarts Age
pod/nmstate-cert-manager-5b47d8dddf-5wnb5 1/1 Running 0 77s
pod/nmstate-console-plugin-d6b76c6b9-4dcwm 1/1 Running 0 77s
pod/nmstate-handler-6v7rm 1/1 Running 0 77s
pod/nmstate-handler-bjcxw 1/1 Running 0 77s
pod/nmstate-handler-fv6m2 1/1 Running 0 77s
pod/nmstate-handler-kb8j6 1/1 Running 0 77s
pod/nmstate-handler-wn55p 1/1 Running 0 77s
pod/nmstate-operator-f6bb869b6-v5m92 1/1 Running 0 4m51s
pod/nmstate-webhook-66d6bbd84b-6n674 1/1 Running 0 77s
pod/nmstate-webhook-66d6bbd84b-vlzrd 1/1 Running 0 77s
The Kubernetes NMState Operator, kubernetes-nmstate-operator
, can collect metrics from the kubernetes_nmstate_features_applied
component and expose them as ready-to-use metrics. As a use case for viewing metrics, consider a situation where you created a NodeNetworkConfigurationPolicy
custom resource and you want to confirm that the policy is active.
The |
In the Developer and Administrator perspectives, the Metrics UI includes some predefined CPU, memory, bandwidth, and network packet queries for the selected project. You can run custom Prometheus Query Language (PromQL) queries for CPU, memory, bandwidth, network packet and application metrics for the project.
The following example demonstrates a NodeNetworkConfigurationPolicy
manifest example that is applied to an OKD cluster:
# ...
interfaces:
- name: br1
type: linux-bridge
state: up
ipv4:
enabled: true
dhcp: true
dhcp-custom-hostname: foo
bridge:
options:
stp:
enabled: false
port: []
# ...
The NodeNetworkConfigurationPolicy
manifest exposes metrics and makes them available to the Cluster Monitoring Operator (CMO). The following example shows some exposed metrics:
controller_runtime_reconcile_time_seconds_bucket{controller="nodenetworkconfigurationenactment",le="0.005"} 16
controller_runtime_reconcile_time_seconds_bucket{controller="nodenetworkconfigurationenactment",le="0.01"} 16
controller_runtime_reconcile_time_seconds_bucket{controller="nodenetworkconfigurationenactment",le="0.025"} 16
...
# HELP kubernetes_nmstate_features_applied Number of nmstate features applied labeled by its name
# TYPE kubernetes_nmstate_features_applied gauge
kubernetes_nmstate_features_applied{name="dhcpv4-custom-hostname"} 1
You have installed the OpenShift CLI (oc
).
You have logged in to the web console as the administrator and installed the Kubernetes NMState Operator.
You have access to the cluster as a developer or as a user with view permissions for the project that you are viewing metrics for.
You have enabled monitoring for user-defined projects.
You have deployed a service in a user-defined project.
You have created a NodeNetworkConfigurationPolicy
manifest and applied it to your cluster.
If you want to view the metrics from the Developer perspective in the OKD web console, complete the following tasks:
Click Observe.
To view the metrics of a specific project, select the project in the Project: list. For example, openshift-nmstate
.
Click the Metrics tab.
To visualize the metrics on the plot, select a query from the Select query list or create a custom PromQL query based on the selected query by selecting Show PromQL.
In the Developer perspective, you can only run one query at a time. |
If you want to view the metrics from the Administrator perspective in the OKD web console, complete the following tasks:
Click Observe → Metrics.
Enter kubernetes_nmstate_features_applied
in the Expression field.
Click Add query and then Run queries.
To explore the visualized metrics, do any of the following tasks:
To zoom into the plot and change the time range, do any of the following tasks:
To visually select the time range, click and drag on the plot horizontally.
To select the time range, use the menu which is in the upper left of the console.
To reset the time range, select Reset zoom.
To display the output for all the queries at a specific point in time, hold the mouse cursor on the plot at that point. The query output displays in a pop-up box.