1. Documentation
×

Control plane configuration options for Amazon Web Services

Sample YAML for configuring Amazon Web Services clusters

The following example YAML snippets show provider specification and failure domain configurations for an AWS cluster.

Sample AWS provider specification

When you create a control plane machine set for an existing cluster, the provider specification must match the providerSpec configuration in the control plane machine custom resource (CR) that is created by the installation program. You can omit any field that is set in the failure domain section of the CR.

In the following example, <cluster_id> is the infrastructure ID that is based on the cluster ID that you set when you provisioned the cluster. If you have the OpenShift CLI installed, you can obtain the infrastructure ID by running the following command:

$ oc get -o jsonpath='{.status.infrastructureName}{"\n"}' infrastructure cluster
Sample AWS providerSpec values
apiVersion: machine.openshift.io/v1
kind: ControlPlaneMachineSet
metadata:
  name: cluster
  namespace: openshift-machine-api
spec:
# ...
  template:
# ...
      spec:
        providerSpec:
          value:
            ami:
              id: ami-<ami_id_string> (1)
            apiVersion: machine.openshift.io/v1beta1
            blockDevices:
            - ebs: (2)
                encrypted: true
                iops: 0
                kmsKey:
                  arn: ""
                volumeSize: 120
                volumeType: gp3
            credentialsSecret:
              name: aws-cloud-credentials (3)
            deviceIndex: 0
            iamInstanceProfile:
              id: <cluster_id>-master-profile (4)
            instanceType: m6i.xlarge (5)
            kind: AWSMachineProviderConfig (6)
            loadBalancers: (7)
            - name: <cluster_id>-int
              type: network
            - name: <cluster_id>-ext
              type: network
            metadata:
              creationTimestamp: null
            metadataServiceOptions: {}
            placement: (8)
              region: <region> (9)
              availabilityZone: "" (10)
              tenancy: (11)
            securityGroups:
            - filters:
              - name: tag:Name
                values:
                - <cluster_id>-master-sg (12)
            subnet: {} (13)
            userDataSecret:
              name: master-user-data (14)
1 Specifies the Fedora CoreOS (FCOS) Amazon Machine Images (AMI) ID for the cluster. The AMI must belong to the same region as the cluster. If you want to use an AWS Marketplace image, you must complete the OKD subscription from the AWS Marketplace to obtain an AMI ID for your region.
2 Specifies the configuration of an encrypted EBS volume.
3 Specifies the secret name for the cluster. Do not change this value.
4 Specifies the AWS Identity and Access Management (IAM) instance profile. Do not change this value.
5 Specifies the AWS instance type for the control plane.
6 Specifies the cloud provider platform type. Do not change this value.
7 Specifies the internal (int) and external (ext) load balancers for the cluster.

You can omit the external (ext) load balancer parameters on private OKD clusters.
8 Specifies where to create the control plane instance in AWS.
9 Specifies the AWS region for the cluster.
10 This parameter is configured in the failure domain and is shown with an empty value here. If a value specified for this parameter differs from the value in the failure domain, the Control Plane Machine Set Operator overwrites it with the value in the failure domain.
11 Specifies the AWS Dedicated Instance configuration for the control plane. For more information, see AWS documentation about Dedicated Instances. The following values are valid:

  • default: The Dedicated Instance runs on shared hardware.

  • dedicated: The Dedicated Instance runs on single-tenant hardware.

  • host: The Dedicated Instance runs on a Dedicated Host, which is an isolated server with configurations that you can control.
12 Specifies the control plane machines security group.
13 This parameter is configured in the failure domain and is shown with an empty value here. If a value specified for this parameter differs from the value in the failure domain, the Control Plane Machine Set Operator overwrites it with the value in the failure domain.

If the failure domain configuration does not specify a value, the value in the provider specification is used. Configuring a subnet in the failure domain overwrites the subnet value in the provider specification.
14 Specifies the control plane user data secret. Do not change this value.

Sample AWS failure domain configuration

The control plane machine set concept of a failure domain is analogous to existing AWS concept of an Availability Zone (AZ). The ControlPlaneMachineSet CR spreads control plane machines across multiple failure domains when possible.

When configuring AWS failure domains in the control plane machine set, you must specify the availability zone name and the subnet to use.

Sample AWS failure domain values
apiVersion: machine.openshift.io/v1
kind: ControlPlaneMachineSet
metadata:
  name: cluster
  namespace: openshift-machine-api
spec:
# ...
  template:
# ...
    machines_v1beta1_machine_openshift_io:
      failureDomains:
        aws:
        - placement:
            availabilityZone: <aws_zone_a> (1)
          subnet: (2)
            filters:
            - name: tag:Name
              values:
              - <cluster_id>-private-<aws_zone_a> (3)
            type: Filters (4)
        - placement:
            availabilityZone: <aws_zone_b> (5)
          subnet:
            filters:
            - name: tag:Name
              values:
              - <cluster_id>-private-<aws_zone_b> (6)
            type: Filters
        platform: AWS (7)
# ...
1 Specifies an AWS availability zone for the first failure domain.
2 Specifies a subnet configuration. In this example, the subnet type is Filters, so there is a filters stanza.
3 Specifies the subnet name for the first failure domain, using the infrastructure ID and the AWS availability zone.
4 Specifies the subnet type. The allowed values are: ARN, Filters and ID. The default value is Filters.
5 Specifies the subnet name for an additional failure domain, using the infrastructure ID and the AWS availability zone.
6 Specifies the cluster’s infrastructure ID and the AWS availability zone for the additional failure domain.
7 Specifies the cloud provider platform name. Do not change this value.

Enabling Amazon Web Services features for control plane machines

You can enable features by updating values in the control plane machine set.

Restricting the API server to private

After you deploy a cluster to Amazon Web Services (AWS), you can reconfigure the API server to use only the private zone.

Prerequisites

  • Install the OpenShift CLI (oc).

  • Have access to the web console as a user with admin privileges.

Procedure

  1. In the web portal or console for your cloud provider, take the following actions:

    1. Locate and delete the appropriate load balancer component:

      • For AWS, delete the external load balancer. The API DNS entry in the private zone already points to the internal load balancer, which uses an identical configuration, so you do not need to modify the internal load balancer.

    2. Delete the api.$clustername.$yourdomain DNS entry in the public zone.

  2. Remove the external load balancers by deleting the following indicated lines in the control plane machine set custom resource:

    # ...
providerSpec:
  value:
# ...
    loadBalancers:
    - name: lk4pj-ext (1)
      type: network (2)
    - name: lk4pj-int
      type: network
# ...
    1 Delete the name value for the external load balancer, which ends in -ext.
    2 Delete the type value for the external load balancer.
Additional resources

Changing the Amazon Web Services instance type by using a control plane machine set

You can change the Amazon Web Services (AWS) instance type that your control plane machines use by updating the specification in the control plane machine set custom resource (CR).

Prerequisites

  • Your AWS cluster uses a control plane machine set.

Procedure

  1. Edit the following line under the providerSpec field:

    providerSpec:
  value:
    ...
    instanceType: <compatible_aws_instance_type> (1)
    1 Specify a larger AWS instance type with the same base as the previous selection. For example, you can change m6i.xlarge to m6i.2xlarge or m6i.4xlarge.

  2. Save your changes.

Assigning machines to placement groups for Elastic Fabric Adapter instances by using machine sets

You can configure a machine set to deploy machines on Elastic Fabric Adapter (EFA) instances within an existing AWS placement group.

EFA instances do not require placement groups, and you can use placement groups for purposes other than configuring an EFA. This example uses both to demonstrate a configuration that can improve network performance for machines within the specified placement group.

Prerequisites

  • You created a placement group in the AWS console.

    Ensure that the rules and limitations for the type of placement group that you create are compatible with your intended use case. The control plane machine set spreads the control plane machines across multiple failure domains when possible. To use placement groups for the control plane, you must use a placement group type that can span multiple Availability Zones.
Procedure

  1. In a text editor, open the YAML file for an existing machine set or create a new one.

  2. Edit the following lines under the providerSpec field:

    apiVersion: machine.openshift.io/v1
kind: ControlPlaneMachineSet
# ...
spec:
  template:
    spec:
      providerSpec:
        value:
          instanceType: <supported_instance_type> (1)
          networkInterfaceType: EFA (2)
          placement:
            availabilityZone: <zone> (3)
            region: <region> (4)
          placementGroupName: <placement_group> (5)
# ...
    1 Specify an instance type that supports EFAs.
    2 Specify the EFA network interface type.
    3 Specify the zone, for example, us-east-1a.
    4 Specify the region, for example, us-east-1.
    5 Specify the name of the existing AWS placement group to deploy machines in.
Verification

  • In the AWS console, find a machine that the machine set created and verify the following in the machine properties:

    • The placement group field has the value that you specified for the placementGroupName parameter in the machine set.

    • The interface type field indicates that it uses an EFA.

Machine set options for the Amazon EC2 Instance Metadata Service

You can use machine sets to create machines that use a specific version of the Amazon EC2 Instance Metadata Service (IMDS). Machine sets can create machines that allow the use of both IMDSv1 and IMDSv2 or machines that require the use of IMDSv2.

Using IMDSv2 is only supported on AWS clusters that were created with OKD version 4.7 or later.

Before configuring a machine set to create machines that require IMDSv2, ensure that any workloads that interact with the AWS metadata service support IMDSv2.

Configuring IMDS by using machine sets

You can specify whether to require the use of IMDSv2 by adding or editing the value of metadataServiceOptions.authentication in the machine set YAML file for your machines.

Prerequisites

  • To use IMDSv2, your AWS cluster must have been created with OKD version 4.7 or later.

Procedure

  • Add or edit the following lines under the providerSpec field:

    providerSpec:
  value:
    metadataServiceOptions:
      authentication: Required (1)
    1 To require IMDSv2, set the parameter value to Required. To allow the use of both IMDSv1 and IMDSv2, set the parameter value to Optional. If no value is specified, both IMDSv1 and IMDSv2 are allowed.

Machine sets that deploy machines as Dedicated Instances

You can create a machine set running on AWS that deploys machines as Dedicated Instances. Dedicated Instances run in a virtual private cloud (VPC) on hardware that is dedicated to a single customer. These Amazon EC2 instances are physically isolated at the host hardware level. The isolation of Dedicated Instances occurs even if the instances belong to different AWS accounts that are linked to a single payer account. However, other instances that are not dedicated can share hardware with Dedicated Instances if they belong to the same AWS account.

Instances with either public or dedicated tenancy are supported by the Machine API. Instances with public tenancy run on shared hardware. Public tenancy is the default tenancy. Instances with dedicated tenancy run on single-tenant hardware.

Creating Dedicated Instances by using machine sets

You can run a machine that is backed by a Dedicated Instance by using Machine API integration. Set the tenancy field in your machine set YAML file to launch a Dedicated Instance on AWS.

Procedure

  • Specify a dedicated tenancy under the providerSpec field:

    providerSpec:
  placement:
    tenancy: dedicated