-
tcp/22 from host running the installer/Ansible
×
Configuring for Amazon Web Services (AWS)
Overview
OKD can be configured to access an AWS EC2 infrastructure, including using AWS volumes as persistent storage for application data. After you configure AWS, some additional configurations must be completed on the OKD hosts.
Permissions
Configuring AWS for OKD requires the following permissions:
Elastic Compute Cloud(EC2) |
|
Elastic Load Balancing |
|
Elastic Compute Cloud(EC2) |
|
|
Configuring a Security Group
When installing OKD on AWS, ensure that you set up the appropriate security groups.
These are some ports that you must have in your security groups, without which the installation fails. You may need more depending on the cluster configuration you want to install. For more information and to adjust your security groups accordingly, see Required Ports for more information.
All OKD Hosts |
|
|---|---|
etcd Security Group |
|
Master Security Group |
|
Node Security Group |
|
Infrastructure Nodes (ones that can host the OKD router) |
|
If configuring external load-balancers (ELBs) for load balancing the masters and/or routers, you also need to configure Ingress and Egress security groups for the ELBs appropriately.
Overriding Detected IP Addresses and Host Names
In AWS, situations that require overriding the variables include:
| Variable | Usage |
|---|---|
|
The user is installing in a VPC that is not configured for both |
|
You have multiple network interfaces configured and want to
use one other than the default. You must also set the
|
|
|
|
|
|
If |
For EC2 hosts in particular, they must be deployed in a VPC that has both
DNS host names and DNS resolution enabled, and openshift_hostname
should not be overridden.
Configuring AWS Variables
To set the required AWS variables, create a /etc/origin/cloudprovider/aws.conf file with the following contents on all of your OKD hosts, both masters and nodes:
[Global] Zone = us-east-1c (1)
| 1 | This is the Availability Zone of your AWS Instance and where your EBS Volume resides; this information is obtained from the AWS Management Console. |
Configuring OKD for AWS
You can set the AWS configuration on OKD in two ways:
Configuring OKD for AWS with Ansible
During
advanced installations,
AWS can be configured using
the openshift_cloudprovider_aws_access_key, openshift_cloudprovider_aws_secret_key, openshift_cloudprovider_kind, openshift_clusterid parameters, which are configurable in the inventory file.
Example AWS Configuration with Ansible
# Cloud Provider Configuration
#
# Note: You may make use of environment variables rather than store
# sensitive configuration within the ansible inventory.
# For example:
#openshift_cloudprovider_aws_access_key="{{ lookup('env','AWS_ACCESS_KEY_ID') }}"
#openshift_cloudprovider_aws_secret_key="{{ lookup('env','AWS_SECRET_ACCESS_KEY') }}"
#
#openshift_clusterid=unique_identifier_per_availablility_zone
#
# AWS (Using API Credentials)
#openshift_cloudprovider_kind=aws
#openshift_cloudprovider_aws_access_key=aws_access_key_id
#openshift_cloudprovider_aws_secret_key=aws_secret_access_key
#
# AWS (Using IAM Profiles)
#openshift_cloudprovider_kind=aws
# Note: IAM roles must exist before launching the instances.
|
When Ansible configures AWS, it automatically makes the necessary changes to the following files:
|
Manually Configuring OKD Masters for AWS
Edit or
create
the master configuration file on all masters
(/etc/origin/master/master-config.yaml by default) and update the contents
of the apiServerArguments and controllerArguments sections:
kubernetesMasterConfig:
...
apiServerArguments:
cloud-provider:
- "aws"
cloud-config:
- "/etc/origin/cloudprovider/aws.conf"
controllerArguments:
cloud-provider:
- "aws"
cloud-config:
- "/etc/origin/cloudprovider/aws.conf"Currently, the nodeName must match the instance name in AWS in order
for the cloud provider integration to work properly. The name must also be
RFC1123 compliant.
|
When triggering a containerized installation, only the directories of /etc/origin and /var/lib/origin are mounted to the master and node container. Therefore, aws.conf should be in /etc/origin/ instead of /etc/. |
Manually Configuring OKD Nodes for AWS
Edit or
create
the node configuration file on all nodes (/etc/origin/node/node-config.yaml
by default) and update the contents of the kubeletArguments section:
kubeletArguments:
cloud-provider:
- "aws"
cloud-config:
- "/etc/origin/cloudprovider/aws.conf"|
When triggering a containerized installation, only the directories of /etc/origin and /var/lib/origin are mounted to the master and node container. Therefore, aws.conf should be in /etc/origin/ instead of /etc/. |
Manually Setting Key-Value Access Pairs
Make sure the following environment variables are set in the /etc/sysconfig/origin-master-api file and /etc/sysconfig/origin-master-controllers file on masters and the /etc/sysconfig/origin-node file on nodes:
AWS_ACCESS_KEY_ID=<key_ID> AWS_SECRET_ACCESS_KEY=<secret_key>
|
Access keys are obtained when setting up your AWS IAM user. |
Applying Configuration Changes
Start or restart OKD services on all master and node hosts to apply your configuration changes, see Restarting OKD services:
# systemctl restart origin-master-api origin-master-controllers # systemctl restart origin-node
Switching from not using a cloud provider to using a cloud provider produces an
error message. Adding the cloud provider tries to delete the node because the
node switches from using the hostname as the externalID (which would have
been the case when no cloud provider was being used) to using the cloud
provider’s instance-id (which is what the cloud provider specifies). To
resolve this issue:
-
Log in to the CLI as a cluster administrator.
-
Check and back up existing node labels:
$ oc describe node <node_name> | grep -Poz '(?s)Labels.*\n.*(?=Taints)' -
Delete the nodes:
$ oc delete node <node_name> -
On each node host, restart the OKD service.
# systemctl restart origin-node
-
Add back any labels on each node that you previously had.
Labeling Clusters for AWS
Starting with OKD version 3.7 of the atomic-openshift-installer,
if you configured AWS provider credentials, you must also ensure that all
hosts are labeled.
To correctly identify which resources are associated with a cluster, tag
resources with the key kubernetes.io/cluster/<clusterid>, where:
-
<clusterid>is a unique name for the cluster.
Set the corresponding value to owned if the node belongs exclusively to the
cluster or to shared if it is a resource shared with other systems.
Tagging all resources with the kubernetes.io/cluster/<clusterid>,Value=(owned|shared)
tag avoids potential issues with multiple zones or multiple clusters.
|
In versions prior to OKD version 3.6, this was
|
See Pods and Services to learn more about labeling and tagging in OKD.
Resources That Need Tags
There are four types of resources that need to be tagged:
-
Instances
-
Security Groups
-
Load Balancers
-
EBS Volumes
Tagging an Existing Cluster
A cluster uses the value of the
kubernetes.io/cluster/<clusterid>,Value=(owned|shared) tag to determine which
resources belong to the AWS cluster. This means that all relevant resources must
be labeled with the kubernetes.io/cluster/<clusterid>,Value=(owned|shared)
tag using the same values for that key. These resources include:
-
All hosts.
-
All relevant load balancers to be used in the AWS instances.
-
All EBS volumes. The EBS Volumes that need to be tagged can found with:
$ oc get pv -o json|jq '.items[].spec.awsElasticBlockStore.volumeID' -
All relevant security groups to be used with the AWS instances.
Do not tag all existing security groups with the
kubernetes.io/cluster/<name>,Value=<clusterid>tag, or the Elastic Load Balancing (ELB) will not be able to create a load balancer.
After tagging any resources, restart the master services on the master and the node service on all nodes. See the Applying Configuration Section.