kubeletArguments: pods-per-core: - "10"
In addition to pod traffic, the most-used data-path in an OKD infrastructure is between the OKD master hosts and etcd. The OKD API server (part of the master binary) consults etcd for node status, network configuration, secrets, and more.
Optimize this traffic path by:
Co-locating master hosts and etcd servers.
Ensuring an uncongested, low latency LAN communication link between master hosts.
The OKD node configuration file at /etc/origin/node/node-config.yaml contains important options, such as the iptables synchronization period, the Maximum Transmission Unit (MTU) of the SDN network, and the proxy-mode.
The node configuration file allows you to pass arguments to the kubelet
(node) process. You can view a list of possible options by running
Not all kubelet options are supported by OKD, and are used in the upstream Kubernetes. This means certain options are in limited support.
In the /etc/origin/node/node-config.yaml file, two parameters control the
maximum number of pods that can be scheduled to a node:
max-pods. When both options are in use, the lower of the two limits the number
of pods on a node. Exceeding these values can result in:
Increased CPU utilization on both OKD and Docker.
Slow pod scheduling.
Potential out-of-memory scenarios (depends on the amount of memory in the node).
Exhausting the pool of IP addresses.
Resource overcommitting, leading to poor user application performance.
In Kubernetes, a pod that is holding a single container actually uses two containers. The second container is used to set up networking prior to the actual container starting. Therefore, a system running 10 pods will actually have 20 containers running.
pods-per-core sets the number of pods the node can run based on the number of
processor cores on the node. For example, if
pods-per-core is set to
a node with 4 processor cores, the maximum number of pods allowed on the node
will be 40.
kubeletArguments: pods-per-core: - "10"
max-pods sets the number of pods the node can run to a fixed value, regardless
of the properties of the node.
kubeletArguments: max-pods: - "250"
Using the above example, the default value for
10 and the
default value for
250. This means that unless the node has 25
cores or more, by default,
pods-per-core will be the limiting factor.
See the Sizing Considerations section in the installation documentation for the recommended limits for an OKD cluster. The recommended sizing accounts for OKD and Docker coordination for container status updates. This coordination puts CPU pressure on the master and docker processes, which can include writing a large amount of log data.
etcd is a distributed key-value store that OKD uses for configuration.
After profiling etcd under OKD, etcd frequently performs small amounts of storage input and output. Using etcd with storage that handles small read/write operations quickly, such as SSD, is highly recommended.
Optimize communication between etcd and master hosts either by co-locating them on the same host, or providing a dedicated network.
etcd hosts are typically memory intensive.
Tuned is a tuning profile delivery mechanism enabled by default in Red Hat Enterprise Linux and other Red Hat products. Tuned customizes Linux settings, such as sysctls, power management, and kernel command line options, to optimize the operating system for different workload performance and scalability requirements.
OKD leverages the
tuned daemon and includes Tuned profiles called
atomic-openshift-guest. These profiles safely
increase some of the commonly encountered vertical scaling limits present in the
kernel, and are automatically applied to your system during installation.
The Tuned profiles support inheritance between profiles. On an OKD
system, the findings delivered by Tuned will be the union of
throughput-performance (the default for RHEL) and
Tuned will determine if you are running OKD on a virtual machine,
and, if so, automatically apply
virtual-guest tuning as well.
To see which Tuned profile is enabled on your system, run:
# tuned-adm active Current active profile: atomic-openshift-node-guest
See the Red Hat Enterprise Linux Performance Tuning Guide for more information about Tuned.