Route configuration

Configuring route timeouts
Enabling HTTP strict transport security
Troubleshooting throughput issues
Using cookies to keep route statefulness
- Annotating a route with a cookie
Path-based routes
Route-specific annotations
Configuring the route admission policy
Creating a route through an Ingress object
Configuring the OKD Ingress Controller for dual-stack networking

Configuring route timeouts

You can configure the default timeouts for an existing route when you have services in need of a low timeout, which is required for Service Level Availability (SLA) purposes, or a high timeout, for cases with a slow back end.

Prerequisites

You need a deployed Ingress Controller on a running cluster.

Procedure

Using the oc annotate command, add the timeout to the route:
```
$ oc annotate route <route_name> \
    --overwrite haproxy.router.openshift.io/timeout=<timeout><time_unit> (1)
```
1 Supported time units are microseconds (us), milliseconds (ms), seconds (s), minutes (m), hours (h), or days (d).

The following example sets a timeout of two seconds on a route named myroute:
```
$ oc annotate route myroute --overwrite haproxy.router.openshift.io/timeout=2s
```

Enabling HTTP strict transport security

HTTP Strict Transport Security (HSTS) policy is a security enhancement, which ensures that only HTTPS traffic is allowed on the host. Any HTTP requests are dropped by default. This is useful for ensuring secure interactions with websites, or to offer a secure application for the user’s benefit.

When HSTS is enabled, HSTS adds a Strict Transport Security header to HTTPS responses from the site. You can use the insecureEdgeTerminationPolicy value in a route to redirect to send HTTP to HTTPS. However, when HSTS is enabled, the client changes all requests from the HTTP URL to HTTPS before the request is sent, eliminating the need for a redirect. This is not required to be supported by the client, and can be disabled by setting max-age=0.

HSTS works only with secure routes (either edge terminated or re-encrypt). The configuration is ineffective on HTTP or passthrough routes.

Procedure

To enable HSTS on a route, add the haproxy.router.openshift.io/hsts_header value to the edge terminated or re-encrypt route:

apiVersion: v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/hsts_header: max-age=31536000;includeSubDomains;preload   (1) (2) (3)

1	`max-age` is the only required parameter. It measures the length of time, in seconds, that the HSTS policy is in effect. The client updates `max-age` whenever a response with a HSTS header is received from the host. When `max-age` times out, the client discards the policy.
2	`includeSubDomains` is optional. When included, it tells the client that all subdomains of the host are to be treated the same as the host.
3	`preload` is optional. When `max-age` is greater than 0, then including `preload` in `haproxy.router.openshift.io/hsts_header` allows external services to include this site in their HSTS preload lists. For example, sites such as Google can construct a list of sites that have `preload` set. Browsers can then use these lists to determine which sites they can communicate with over HTTPS, before they have interacted with the site. Without `preload` set, browsers must have interacted with the site over HTTPS to get the header.

Troubleshooting throughput issues

Sometimes applications deployed through OKD can cause network throughput issues such as unusually high latency between specific services.

Use the following methods to analyze performance issues if pod logs do not reveal any cause of the problem:

Use a packet analyzer, such as ping or tcpdump to analyze traffic between a pod and its node.

For example, run the tcpdump tool on each pod while reproducing the behavior that led to the issue. Review the captures on both sides to compare send and receive timestamps to analyze the latency of traffic to and from a pod. Latency can occur in OKD if a node interface is overloaded with traffic from other pods, storage devices, or the data plane.
```
$ tcpdump -s 0 -i any -w /tmp/dump.pcap host <podip 1> && host <podip 2> (1)
```
1 podip is the IP address for the pod. Run the oc get pod <pod_name> -o wide command to get the IP address of a pod.

tcpdump generates a file at /tmp/dump.pcap containing all traffic between these two pods. Ideally, run the analyzer shortly before the issue is reproduced and stop the analyzer shortly after the issue is finished reproducing to minimize the size of the file. You can also run a packet analyzer between the nodes (eliminating the SDN from the equation) with:
```
$ tcpdump -s 0 -i any -w /tmp/dump.pcap port 4789
```
Use a bandwidth measuring tool, such as iperf, to measure streaming throughput and UDP throughput. Run the tool from the pods first, then from the nodes, to locate any bottlenecks.

Using cookies to keep route statefulness

OKD provides sticky sessions, which enables stateful application traffic by ensuring all traffic hits the same endpoint. However, if the endpoint pod terminates, whether through restart, scaling, or a change in configuration, this statefulness can disappear.

OKD can use cookies to configure session persistence. The Ingress controller selects an endpoint to handle any user requests, and creates a cookie for the session. The cookie is passed back in the response to the request and the user sends the cookie back with the next request in the session. The cookie tells the Ingress Controller which endpoint is handling the session, ensuring that client requests use the cookie so that they are routed to the same pod.

You can set a cookie name to overwrite the default, auto-generated one for the route. This allows the application receiving route traffic to know the cookie name. By deleting the cookie it can force the next request to re-choose an endpoint. So, if a server was overloaded it tries to remove the requests from the client and redistribute them.

Procedure

Annotate the route with the specified cookie name:
```
$ oc annotate route <route_name> router.openshift.io/cookie_name="<cookie_name>"
```
where:

<route_name>

Specifies the name of the route.

<cookie_name>

Specifies the name for the cookie.

For example, to annotate the route my_route with the cookie name my_cookie:
```
$ oc annotate route my_route router.openshift.io/cookie_name="my_cookie"
```
Capture the route host name in a variable:
```
$ ROUTE_NAME=$(oc get route <route_name> -o jsonpath='{.spec.host}')
```
where:

<route_name>

Specifies the name of the route.
Save the cookie, and then access the route:
```
$ curl $ROUTE_NAME -k -c /tmp/cookie_jar
```
Use the cookie saved by the previous command when connecting to the route:
```
$ curl $ROUTE_NAME -k -b /tmp/cookie_jar
```

Path-based routes

Path-based routes specify a path component that can be compared against a URL, which requires that the traffic for the route be HTTP based. Thus, multiple routes can be served using the same host name, each with a different path. Routers should match routes based on the most specific path to the least. However, this depends on the router implementation.

The following table shows example routes and their accessibility:

Table 1. Route availability
Route	When Compared to	Accessible
www.example.com/test	www.example.com/test	Yes
www.example.com/test	www.example.com	No
www.example.com/test and www.example.com	www.example.com/test	Yes
www.example.com/test and www.example.com	www.example.com	Yes
www.example.com	www.example.com/text	Yes (Matched by the host, not the route)
www.example.com	www.example.com	Yes

An unsecured route with a path

apiVersion: v1
kind: Route
metadata:
  name: route-unsecured
spec:
  host: www.example.com
  path: "/test" (1)
  to:
    kind: Service
    name: service-name

1	The path is the only added attribute for a path-based route.

Path-based routing is not available when using passthrough TLS, as the router does not terminate TLS in that case and cannot read the contents of the request.

Route-specific annotations

The Ingress Controller can set the default options for all the routes it exposes. An individual route can override some of these defaults by providing specific configurations in its annotations. Red Hat does not support adding a route annotation to an operator-managed route.

To create a whitelist with multiple source IPs or subnets, use a space-delimited list. Any other delimiter type causes the list to be ignored without a warning or error message.

Table 2. Route annotations
Variable	Description	Environment variable used as default
`haproxy.router.openshift.io/balance`	Sets the load-balancing algorithm. Available options are `random`, `source`, `roundrobin`, and `leastconn`.	`ROUTER_TCP_BALANCE_SCHEME` for passthrough routes. Otherwise, use `ROUTER_LOAD_BALANCE_ALGORITHM`.
`haproxy.router.openshift.io/disable_cookies`	Disables the use of cookies to track related connections. If set to `true` or `TRUE`, the balance algorithm is used to choose which back-end serves connections for each incoming HTTP request.
`router.openshift.io/cookie_name`	Specifies an optional cookie to use for this route. The name must consist of any combination of upper and lower case letters, digits, "_", and "-". The default is the hashed internal key name for the route.
`haproxy.router.openshift.io/pod-concurrent-connections`	Sets the maximum number of connections that are allowed to a backing pod from a router. Note: if there are multiple pods, each can have this many connections. But if you have multiple routers, there is no coordination among them, each may connect this many times. If not set, or set to 0, there is no limit.
`haproxy.router.openshift.io/rate-limit-connections`	Setting `true` or `TRUE` to enables rate limiting functionality.
`haproxy.router.openshift.io/rate-limit-connections.concurrent-tcp`	Limits the number of concurrent TCP connections shared by an IP address.
`haproxy.router.openshift.io/rate-limit-connections.rate-http`	Limits the rate at which an IP address can make HTTP requests.
`haproxy.router.openshift.io/rate-limit-connections.rate-tcp`	Limits the rate at which an IP address can make TCP connections.
`haproxy.router.openshift.io/timeout`	Sets a server-side timeout for the route. (TimeUnits)	`ROUTER_DEFAULT_SERVER_TIMEOUT`
`router.openshift.io/haproxy.health.check.interval`	Sets the interval for the back-end health checks. (TimeUnits)	`ROUTER_BACKEND_CHECK_INTERVAL`
`haproxy.router.openshift.io/ip_whitelist`	Sets a whitelist for the route. The whitelist is a space-separated list of IP addresses and CIDR ranges for the approved source addresses. Requests from IP addresses that are not in the whitelist are dropped. The maximum number of IP addresses and CIDR ranges allowed in a whitelist is 61.
`haproxy.router.openshift.io/hsts_header`	Sets a Strict-Transport-Security header for the edge terminated or re-encrypt route.
`haproxy.router.openshift.io/log-send-hostname`	Sets the `hostname` field in the Syslog header. Uses the host name of the system. `log-send-hostname` is enabled by default if any Ingress API logging method, such as sidecar or Syslog facility, is enabled for the router.
`haproxy.router.openshift.io/rewrite-target`	Sets the rewrite path of the request on the backend.
`haproxy.router.openshift.io/cookie-same-site`	Sets a value to restrict cookies. The values are: `Lax`: cookies are transferred between the visited site and third-party sites. `Strict`: cookies are restricted to the visited site. `None`: cookies are restricted to the visited site. This value is applicable to re-encrypt and edge routes only. For more information, see the SameSite cookies documentation.
`haproxy.router.openshift.io/set-forwarded-headers`	Sets the policy for handling the `Forwarded` and `X-Forwarded-For` HTTP headers per route. The values are: `append`: appends the header, preserving any existing header. This is the default value. `replace`: sets the header, removing any existing header. `never`: never sets the header, but preserves any existing header. `if-none`: sets the header if it is not already set.	`ROUTER_SET_FORWARDED_HEADERS`

Environment variables cannot be edited.

A route setting custom timeout

apiVersion: v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/timeout: 5500ms (1)
...

1	Specifies the new timeout with HAProxy supported units (`us`, `ms`, `s`, `m`, `h`, `d`). If the unit is not provided, `ms` is the default.

Setting a server-side timeout value for passthrough routes too low can cause WebSocket connections to timeout frequently on that route.

A route that allows only one specific IP address

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.10

A route that allows several IP addresses

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.10 192.168.1.11 192.168.1.12

A route that allows an IP address CIDR network

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 192.168.1.0/24

A route that allows both IP an address and IP address CIDR networks

metadata:
  annotations:
    haproxy.router.openshift.io/ip_whitelist: 180.5.61.153 192.168.1.0/24 10.0.0.0/8

A route specifying a rewrite target

apiVersion: v1
kind: Route
metadata:
  annotations:
    haproxy.router.openshift.io/rewrite-target: / (1)
...

1	Sets `/` as rewrite path of the request on the backend.

Setting the haproxy.router.openshift.io/rewrite-target annotation on a route specifies that the Ingress Controller should rewrite paths in HTTP requests using this route before forwarding the requests to the backend application. The part of the request path that matches the path specified in spec.path is replaced with the rewrite target specified in the annotation.

The following table provides examples of the path rewriting behavior for various combinations of spec.path, request path, and rewrite target.

Table 3. rewrite-target examples:
Route.spec.path	Request path	Rewrite target	Forwarded request path
/foo	/foo	/	/
/foo	/foo/	/	/
/foo	/foo/bar	/	/bar
/foo	/foo/bar/	/	/bar/
/foo	/foo	/bar	/bar
/foo	/foo/	/bar	/bar/
/foo	/foo/bar	/baz	/baz/bar
/foo	/foo/bar/	/baz	/baz/bar/
/foo/	/foo	/	N/A (request path does not match route path)
/foo/	/foo/	/	/
/foo/	/foo/bar	/	/bar

Configuring the route admission policy

Administrators and application developers can run applications in multiple namespaces with the same domain name. This is for organizations where multiple teams develop microservices that are exposed on the same host name.

Allowing claims across namespaces should only be enabled for clusters with trust between namespaces, otherwise a malicious user could take over a host name. For this reason, the default admission policy disallows host name claims across namespaces.

Prerequisites

Cluster administrator privileges.

Procedure

Edit the .spec.routeAdmission field of the ingresscontroller resource variable using the following command:

$ oc -n openshift-ingress-operator patch ingresscontroller/default --patch '{"spec":{"routeAdmission":{"namespaceOwnership":"InterNamespaceAllowed"}}}' --type=merge

Sample Ingress Controller configuration

spec:
  routeAdmission:
    namespaceOwnership: InterNamespaceAllowed
...

You can alternatively apply the following YAML to configure the route admission policy:

apiVersion: operator.openshift.io/v1
kind: IngressController
metadata:
  name: default
  namespace: openshift-ingress-operator
spec:
  routeAdmission:
    namespaceOwnership: InterNamespaceAllowed

Creating a route through an Ingress object

Some ecosystem components have an integration with Ingress resources but not with route resources. To cover this case, OKD automatically creates managed route objects when an Ingress object is created. These route objects are deleted when the corresponding Ingress objects are deleted.

Procedure

Define an Ingress object in the OKD console or by entering the oc create command:

YAML Definition of an Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: frontend
  annotations:
    route.openshift.io/termination: "reencrypt" (1)
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - backend:
          serviceName: frontend
          servicePort: 443
  tls:
  - hosts:
    - www.example.com
    secretName: example-com-tls-certificate

The route.openshift.io/termination annotation can be used to configure the spec.tls.termination field of the Route as Ingress has no field for this. The accepted values are edge, passthrough and reencrypt. All other values are silently ignored. When unset, edge is used.

If you specify the passthrough value in the route.openshift.io/termination annotation, set path to '' and pathType to ImplementationSpecific in the spec:

  spec:
    rules:
    - host: www.example.com
      http:
        paths:
        - path: ''
          pathType: ImplementationSpecific
        - backend:
            serviceName: frontend
            servicePort: 443

$ oc apply -f ingress.yaml

List your routes:

$ oc get routes

The result includes an autogenerated route whose name starts with frontend-:

NAME             HOST/PORT         PATH    SERVICES    PORT    TERMINATION          WILDCARD
frontend-gnztq   www.example.com           frontend    443     reencrypt/Redirect   None

If you inspect this route, it looks this:

YAML Definition of an autogenerated route

apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: frontend-gnztq
  ownerReferences:
  - apiVersion: networking.k8s.io/v1
    controller: true
    kind: Ingress
    name: frontend
    uid: 4e6c59cc-704d-4f44-b390-617d879033b6
spec:
  host: www.example.com
  to:
    kind: Service
    name: frontend
  tls:
    certificate: |
      -----BEGIN CERTIFICATE-----
      [...]
      -----END CERTIFICATE-----
    insecureEdgeTerminationPolicy: Redirect
    key: |
      -----BEGIN RSA PRIVATE KEY-----
      [...]
      -----END RSA PRIVATE KEY-----
    termination: reencrypt

Configuring the OKD Ingress Controller for dual-stack networking

If your OKD cluster is configured for IPv4 and IPv6 dual-stack networking, your cluster is is externally reachable by OKD routes.

The Ingress Controller automatically serves services that have both IPv4 and IPv6 endpoints, but you can configure the Ingress Controller for single-stack or dual-stack services.

Prerequisites

You deployed an OKD cluster on bare metal.
You installed the OpenShift CLI (oc).

Procedure

To have the Ingress Controller serve traffic over IPv4/IPv6 to a workload, you can create a service YAML file or modify an existing service YAML file by setting the ipFamilies and ipFamilyPolicy fields. For example:

Sample service YAML file

apiVersion: v1
  kind: Service
  metadata:
    creationTimestamp: yyyy-mm-ddT00:00:00Z
    labels:
      name: <service_name>
      manager: kubectl-create
      operation: Update
      time: yyyy-mm-ddT00:00:00Z
    name: <service_name>
    namespace: <namespace_name>
    resourceVersion: "<resource_version_number>"
    selfLink: "/api/v1/namespaces/<namespace_name>/services/<service_name>"
    uid: <uid_number>
  spec:
    clusterIP: 172.30.0.0/16
    clusterIPs: (1)
    - 172.30.0.0/16
    - <second_IP_address>
    ipFamilies: (2)
    - IPv4
    - IPv6
    ipFamilyPolicy: RequireDualStack (3)
    ports:
    - port: 8080
      protocol: TCP
      targetport: 8080
    selector:
      name: <namespace_name>
    sessionAffinity: None
    type: ClusterIP
  status:
    loadbalancer: {}

1	In a dual-stack instance, there are two different `clusterIPs` provided.
2	For a single-stack instance, enter `IPv4` or `IPv6`. For a dual-stack instance, enter both `IPv4` and `IPv6`.
3	For a single-stack instance, enter `SingleStack`. For a dual-stack instance, enter `RequireDualStack`.

These resources generate corresponding endpoints. The Ingress Controller now watches endpointslices.

To view endpoints, enter the following command:
```
$ oc get endpoints
```
To view endpointslices, enter the following command:
```
$ oc get endpointslices
```

Configuring route timeouts

Enabling HTTP strict transport security

Troubleshooting throughput issues

Using cookies to keep route statefulness

Annotating a route with a cookie

Path-based routes

Route-specific annotations

Configuring the route admission policy

Creating a route through an Ingress object

Configuring the OKD Ingress Controller for dual-stack networking