Creating secondary networks with other CNI plugins - Multiple networks | Networking

Configuration for a bridge additional network
- Bridge CNI plugin configuration example
Configuration for a host device additional network
- host-device configuration example
Configuration for a VLAN additional network
- VLAN configuration example
Configuration for an IPVLAN additional network
- IPVLAN CNI plugin configuration example
Configuration for a MACVLAN additional network
- MACVLAN CNI plugin configuration example
Configuration for a TAP additional network
- Tap configuration example
- Setting SELinux boolean for the TAP CNI plugin
Configuring routes using the route-override plugin on an additional network
- Route-override plugin configuration example

The specific configuration fields for additional networks are described in the following sections.

Configuration for a bridge additional network

The following object describes the configuration parameters for the Bridge CNI plugin:

Table 1. Bridge CNI plugin JSON configuration object
Field	Type	Description
`cniVersion`	`string`	The CNI specification version. The `0.3.1` value is required.
`name`	`string`	The value for the `name` parameter you provided previously for the CNO configuration.
`type`	`string`	The name of the CNI plugin to configure: `bridge`.
`ipam`	`object`	The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition.
`bridge`	`string`	Optional: Specify the name of the virtual bridge to use. If the bridge interface does not exist on the host, it is created. The default value is `cni0`.
`ipMasq`	`boolean`	Optional: Set to `true` to enable IP masquerading for traffic that leaves the virtual network. The source IP address for all traffic is rewritten to the bridge’s IP address. If the bridge does not have an IP address, this setting has no effect. The default value is `false`.
`isGateway`	`boolean`	Optional: Set to `true` to assign an IP address to the bridge. The default value is `false`.
`isDefaultGateway`	`boolean`	Optional: Set to `true` to configure the bridge as the default gateway for the virtual network. The default value is `false`. If `isDefaultGateway` is set to `true`, then `isGateway` is also set to `true` automatically.
`forceAddress`	`boolean`	Optional: Set to `true` to allow assignment of a previously assigned IP address to the virtual bridge. When set to `false`, if an IPv4 address or an IPv6 address from overlapping subsets is assigned to the virtual bridge, an error occurs. The default value is `false`.
`hairpinMode`	`boolean`	Optional: Set to `true` to allow the virtual bridge to send an Ethernet frame back through the virtual port it was received on. This mode is also known as reflective relay. The default value is `false`.
`promiscMode`	`boolean`	Optional: Set to `true` to enable promiscuous mode on the bridge. The default value is `false`.
`vlan`	`string`	Optional: Specify a virtual LAN (VLAN) tag as an integer value. By default, no VLAN tag is assigned.
`preserveDefaultVlan`	`string`	Optional: Indicates whether the default vlan must be preserved on the `veth` end connected to the bridge. Defaults to true.
`vlanTrunk`	`list`	Optional: Assign a VLAN trunk tag. The default value is `none`.
`mtu`	`integer`	Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
`enabledad`	`boolean`	Optional: Enables duplicate address detection for the container side `veth`. The default value is `false`.
`macspoofchk`	`boolean`	Optional: Enables mac spoof check, limiting the traffic originating from the container to the mac address of the interface. The default value is `false`.

The VLAN parameter configures the VLAN tag on the host end of the veth and also enables the vlan_filtering feature on the bridge interface.

To configure an uplink for an L2 network, you must allow the VLAN on the uplink interface by using the following command:

$  bridge vlan add vid VLAN_ID dev DEV

Bridge CNI plugin configuration example

The following example configures an additional network named bridge-net:

{
  "cniVersion": "0.3.1",
  "name": "bridge-net",
  "type": "bridge",
  "isGateway": true,
  "vlan": 2,
  "ipam": {
    "type": "dhcp"
    }
}

Configuration for a host device additional network

Specify your network device by setting only one of the following parameters: device,hwaddr, kernelpath, or pciBusID.

The following object describes the configuration parameters for the host-device CNI plugin:

Table 2. Host device CNI plugin JSON configuration object
Field	Type	Description
`cniVersion`	`string`	The CNI specification version. The `0.3.1` value is required.
`name`	`string`	The value for the `name` parameter you provided previously for the CNO configuration.
`type`	`string`	The name of the CNI plugin to configure: `host-device`.
`device`	`string`	Optional: The name of the device, such as `eth0`.
`hwaddr`	`string`	Optional: The device hardware MAC address.
`kernelpath`	`string`	Optional: The Linux kernel device path, such as `/sys/devices/pci0000:00/0000:00:1f.6`.
`pciBusID`	`string`	Optional: The PCI address of the network device, such as `0000:00:1f.6`.

host-device configuration example

The following example configures an additional network named hostdev-net:

{
  "cniVersion": "0.3.1",
  "name": "hostdev-net",
  "type": "host-device",
  "device": "eth1"
}

Configuration for a VLAN additional network

The following object describes the configuration parameters for the VLAN, vlan, CNI plugin:

Table 3. VLAN CNI plugin JSON configuration object
Field	Type	Description
`cniVersion`	`string`	The CNI specification version. The `0.3.1` value is required.
`name`	`string`	The value for the `name` parameter you provided previously for the CNO configuration.
`type`	`string`	The name of the CNI plugin to configure: `vlan`.
`master`	`string`	The Ethernet interface to associate with the network attachment. If a `master` is not specified, the interface for the default network route is used.
`vlanId`	`integer`	Set the ID of the `vlan`.
`ipam`	`object`	The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition.
`mtu`	`integer`	Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
`dns`	`integer`	Optional: DNS information to return. For example, a priority-ordered list of DNS nameservers.
`linkInContainer`	`boolean`	Optional: Specifies whether the `master` interface is in the container network namespace or the main network namespace. Set the value to `true` to request the use of a container namespace `master` interface.

A NetworkAttachmentDefinition custom resource definition (CRD) with a vlan configuration can be used only on a single pod in a node because the CNI plugin cannot create multiple vlan subinterfaces with the same vlanId on the same master interface.

VLAN configuration example

The following example demonstrates a vlan configuration with an additional network that is named vlan-net:

{
  "name": "vlan-net",
  "cniVersion": "0.3.1",
  "type": "vlan",
  "master": "eth0",
  "mtu": 1500,
  "vlanId": 5,
  "linkInContainer": false,
  "ipam": {
      "type": "host-local",
      "subnet": "10.1.1.0/24"
  },
  "dns": {
      "nameservers": [ "10.1.1.1", "8.8.8.8" ]
  }
}

Configuration for an IPVLAN additional network

The following object describes the configuration parameters for the IPVLAN, ipvlan, CNI plugin:

Table 4. IPVLAN CNI plugin JSON configuration object
Field	Type	Description
`cniVersion`	`string`	The CNI specification version. The `0.3.1` value is required.
`name`	`string`	The value for the `name` parameter you provided previously for the CNO configuration.
`type`	`string`	The name of the CNI plugin to configure: `ipvlan`.
`ipam`	`object`	The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition. This is required unless the plugin is chained.
`mode`	`string`	Optional: The operating mode for the virtual network. The value must be `l2`, `l3`, or `l3s`. The default value is `l2`.
`master`	`string`	Optional: The Ethernet interface to associate with the network attachment. If a `master` is not specified, the interface for the default network route is used.
`mtu`	`integer`	Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
`linkInContainer`	`boolean`	Optional: Specifies whether the `master` interface is in the container network namespace or the main network namespace. Set the value to `true` to request the use of a container namespace `master` interface.

The ipvlan object does not allow virtual interfaces to communicate with the master interface. Therefore the container is not able to reach the host by using the ipvlan interface. Be sure that the container joins a network that provides connectivity to the host, such as a network supporting the Precision Time Protocol (PTP).
A single master interface cannot simultaneously be configured to use both macvlan and ipvlan.
For IP allocation schemes that cannot be interface agnostic, the ipvlan plugin can be chained with an earlier plugin that handles this logic. If the master is omitted, then the previous result must contain a single interface name for the ipvlan plugin to enslave. If ipam is omitted, then the previous result is used to configure the ipvlan interface.

IPVLAN CNI plugin configuration example

The following example configures an additional network named ipvlan-net:

{
  "cniVersion": "0.3.1",
  "name": "ipvlan-net",
  "type": "ipvlan",
  "master": "eth1",
  "linkInContainer": false,
  "mode": "l3",
  "ipam": {
    "type": "static",
    "addresses": [
       {
         "address": "192.168.10.10/24"
       }
    ]
  }
}

Configuration for a MACVLAN additional network

The following object describes the configuration parameters for the MAC Virtual LAN (MACVLAN) Container Network Interface (CNI) plugin:

Table 5. MACVLAN CNI plugin JSON configuration object
Field	Type	Description
`cniVersion`	`string`	The CNI specification version. The `0.3.1` value is required.
`name`	`string`	The value for the `name` parameter you provided previously for the CNO configuration.
`type`	`string`	The name of the CNI plugin to configure: `macvlan`.
`ipam`	`object`	The configuration object for the IPAM CNI plugin. The plugin manages IP address assignment for the attachment definition.
`mode`	`string`	Optional: Configures traffic visibility on the virtual network. Must be either `bridge`, `passthru`, `private`, or `vepa`. If a value is not provided, the default value is `bridge`.
`master`	`string`	Optional: The host network interface to associate with the newly created macvlan interface. If a value is not specified, then the default route interface is used.
`mtu`	`integer`	Optional: The maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.
`linkInContainer`	`boolean`	Optional: Specifies whether the `master` interface is in the container network namespace or the main network namespace. Set the value to `true` to request the use of a container namespace `master` interface.

If you specify the master key for the plugin configuration, use a different physical network interface than the one that is associated with your primary network plugin to avoid possible conflicts.

MACVLAN CNI plugin configuration example

The following example configures an additional network named macvlan-net:

{
  "cniVersion": "0.3.1",
  "name": "macvlan-net",
  "type": "macvlan",
  "master": "eth1",
  "linkInContainer": false,
  "mode": "bridge",
  "ipam": {
    "type": "dhcp"
    }
}

Configuration for a TAP additional network

The following object describes the configuration parameters for the TAP CNI plugin:

Field Type Description

cniVersion

string

The CNI specification version. The 0.3.1 value is required.

name

string

The value for the name parameter you provided previously for the CNO configuration.

type

string

The name of the CNI plugin to configure: tap.

mac

string

Optional: Request the specified MAC address for the interface.

mtu

integer

Optional: Set the maximum transmission unit (MTU) to the specified value. The default value is automatically set by the kernel.

selinuxcontext

string

Optional: The SELinux context to associate with the tap device.

The value system_u:system_r:container_t:s0 is required for OKD.

multiQueue

boolean

Optional: Set to true to enable multi-queue.

owner

integer

Optional: The user owning the tap device.

group

integer

Optional: The group owning the tap device.

bridge

string

Optional: Set the tap device as a port of an already existing bridge.

Tap configuration example

The following example configures an additional network named mynet:

{
 "name": "mynet",
 "cniVersion": "0.3.1",
 "type": "tap",
 "mac": "00:11:22:33:44:55",
 "mtu": 1500,
 "selinuxcontext": "system_u:system_r:container_t:s0",
 "multiQueue": true,
 "owner": 0,
 "group": 0
 "bridge": "br1"
}

Setting SELinux boolean for the TAP CNI plugin

To create the tap device with the container_t SELinux context, enable the container_use_devices boolean on the host by using the Machine Config Operator (MCO).

Prerequisites

You have installed the OpenShift CLI (oc).

Procedure

Create a new YAML file named, such as setsebool-container-use-devices.yaml, with the following details:

apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: 99-worker-setsebool
spec:
  config:
    ignition:
      version: 3.2.0
    systemd:
      units:
      - enabled: true
        name: setsebool.service
        contents: |
          [Unit]
          Description=Set SELinux boolean for the TAP CNI plugin
          Before=kubelet.service

          [Service]
          Type=oneshot
          ExecStart=/usr/sbin/setsebool container_use_devices=on
          RemainAfterExit=true

          [Install]
          WantedBy=multi-user.target graphical.target

Create the new MachineConfig object by running the following command:
```
$ oc apply -f setsebool-container-use-devices.yaml
```
Applying any changes to the MachineConfig object causes all affected nodes to gracefully reboot after the change is applied. This update can take some time to be applied.

Verify the change is applied by running the following command:

$ oc get machineconfigpools

Expected output

NAME        CONFIG                                                UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE
master      rendered-master-e5e0c8e8be9194e7c5a882e047379cfa      True      False      False      3              3                   3                     0                      7d2h
worker      rendered-worker-d6c9ca107fba6cd76cdcbfcedcafa0f2      True      False      False      3              3                   3                     0                      7d

All nodes should be in the updated and ready state.

Configuring routes using the route-override plugin on an additional network

The following object describes the configuration parameters for the route-override CNI plugin:

Table 7. Route override CNI plugin JSON configuration object
Field	Type	Description
`type`	`string`	The name of the CNI plugin to configure: `route-override`.
`flushroutes`	`boolean`	Optional: Set to `true` to flush any existing routes.
`flushgateway`	`boolean`	Optional: Set to `true` to flush the default route namely the gateway route.
`delroutes`	`object`	Optional: Specify the list of routes to delete from the container namespace.
`addroutes`	`object`	Optional: Specify the list of routes to add to the container namespace. Each route is a dictionary with `dst` and optional `gw` fields. If `gw` is omitted, the plugin uses the default gateway value.
`skipcheck`	`boolean`	Optional: Set this to `true` to skip the check command. By default, CNI plugins verify the network setup during the container lifecycle. When modifying routes dynamically with `route-override`, skipping this check ensures the final configuration reflects the updated routes.

Route-override plugin configuration example

The route-override CNI is a type of CNI that it is designed to be used when chained with a parent CNI. It does not operate independently, but relies on the parent CNI to first create the network interface and assign IP addresses before it can modify the routing rules.

The following example configures an additional network named mymacvlan. The parent CNI creates a network interface attached to eth1 and assigns an IP address in the 192.168.1.0/24 range using host-local IPAM. The route-override CNI is then chained to the parent CNI and modifies the routing rules by flushing existing routes, deleting the route to 192.168.0.0/24, and adding a new route for 192.168.0.0/24 with a custom gateway.

{
    "cniVersion": "0.3.0",
    "name": "mymacvlan",
    "plugins": [
        {
            "type": "macvlan",         (1)
            "master": "eth1",
            "mode": "bridge",
            "ipam": {
                "type": "host-local",
                "subnet": "192.168.1.0/24"
            }
        },
        {
            "type": "route-override",    (2)
            "flushroutes": true,
            "delroutes": [
                {
                    "dst": "192.168.0.0/24"
                }
            ],
            "addroutes": [
                {
                    "dst": "192.168.0.0/24",
                    "gw": "10.1.254.254"
                }
            ]
        }
    ]
}

1	The parent CNI creates a network interface attached to `eth1`.
2	The chained `route-override` CNI modifies the routing rules.

Additional resources

For more information about enabling an SELinux boolean on a node, see Setting SELinux booleans.