Skip to content

AWS: support custom regions and custom endpoints. #3277

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 18 commits into from
Apr 18, 2020
Merged

AWS: support custom regions and custom endpoints. #3277

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
9ca048e
pkg/types/aws: add service endpoints to aws platform
abhinavdahiya Mar 10, 2020
6e32f0d
asset/installconfig/aws/session.go: allow creating session with regio…
abhinavdahiya Mar 11, 2020
5c621c0
asset: Use common session when fetching public zone ID for AWS
abhinavdahiya Mar 11, 2020
915c773
aws: configure terraform with service endpoints
abhinavdahiya Mar 11, 2020
e035875
destroy/aws: use the service overrides for AWS apis
abhinavdahiya Mar 11, 2020
d28c3af
data/aws/main.tf: load sts service endpoint override
abhinavdahiya Mar 11, 2020
805a108
platformtests: drop aws as no longer required
abhinavdahiya Mar 12, 2020
ce9dd4d
aws: use SDK and rhcos AMIs to calculate known regions
abhinavdahiya Mar 12, 2020
92cfcd7
asset/installconfig/aws/platform.go: use the known regions for region…
abhinavdahiya Mar 12, 2020
3e5f419
aws/validation/platform.go: drop region validation from install-config
abhinavdahiya Mar 12, 2020
cf79a75
asset/installconfig/aws: add validation to require endpoints for cert…
abhinavdahiya Mar 12, 2020
0ac6cb1
aws: copy AMI from us-east-1 for SDK regions when not available
abhinavdahiya Mar 13, 2020
982bd2f
vendor: update the openshift/api
abhinavdahiya Mar 16, 2020
204413d
manifests/infrastructure.go: aws: update the config/v1 Infra with ser…
abhinavdahiya Mar 17, 2020
2b4fed0
data/data/aws: skip region validation for custom regions
abhinavdahiya Apr 16, 2020
2ef70a9
aws: add missing endpoints for aws china route53
abhinavdahiya Apr 16, 2020
61ea70e
FIXUP: terraform formatting
abhinavdahiya Apr 17, 2020
6ced6ad
FIXUP: drop shadowing withing range in isAWSRegion
abhinavdahiya Apr 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions data/data/aws/bootstrap/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ resource "aws_instance" "bootstrap" {

tags = merge(
{
"Name" = "${var.cluster_id}-bootstrap"
"Name" = "${var.cluster_id}-bootstrap"
},
var.tags,
)
Expand All @@ -150,7 +150,7 @@ resource "aws_instance" "bootstrap" {

volume_tags = merge(
{
"Name" = "${var.cluster_id}-bootstrap-vol"
"Name" = "${var.cluster_id}-bootstrap-vol"
},
var.tags,
)
Expand All @@ -174,7 +174,7 @@ resource "aws_security_group" "bootstrap" {

tags = merge(
{
"Name" = "${var.cluster_id}-bootstrap-sg"
"Name" = "${var.cluster_id}-bootstrap-sg"
},
var.tags,
)
Expand Down
34 changes: 29 additions & 5 deletions data/data/aws/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,22 @@ locals {
provider "aws" {
region = var.aws_region

# Validation of AWS Bahrain region was added in AWS TF provider v2.22
# so we skip when installing in me-south-1.
skip_region_validation = var.aws_region == "me-south-1"
skip_region_validation = var.aws_skip_region_validation

endpoints {
ec2 = lookup(var.custom_endpoints, "ec2", null)
elb = lookup(var.custom_endpoints, "elasticloadbalancing", null)
iam = lookup(var.custom_endpoints, "iam", null)
route53 = lookup(var.custom_endpoints, "route53", null)
s3 = lookup(var.custom_endpoints, "s3", null)
sts = lookup(var.custom_endpoints, "sts", null)
}
}

module "bootstrap" {
source = "./bootstrap"

ami = var.aws_ami
ami = var.aws_region == var.aws_ami_region ? var.aws_ami : aws_ami_copy.imported[0].id
instance_type = var.aws_bootstrap_instance_type
cluster_id = var.cluster_id
ignition = var.ignition_bootstrap
Expand Down Expand Up @@ -53,7 +60,7 @@ module "masters" {
root_volume_kms_key_id = var.aws_master_root_volume_kms_key_id
target_group_arns = module.vpc.aws_lb_target_group_arns
target_group_arns_length = module.vpc.aws_lb_target_group_arns_length
ec2_ami = var.aws_ami
ec2_ami = var.aws_region == var.aws_ami_region ? var.aws_ami : aws_ami_copy.imported[0].id
user_data_ign = var.ignition_master
publish_strategy = var.aws_publish_strategy
}
Expand Down Expand Up @@ -102,3 +109,20 @@ module "vpc" {
tags = local.tags
}

resource "aws_ami_copy" "imported" {
count = var.aws_region != var.aws_ami_region ? 1 : 0
name = "${var.cluster_id}-master"
source_ami_id = var.aws_ami
source_ami_region = var.aws_ami_region
encrypted = true

tags = merge(
{
"Name" = "${var.cluster_id}-ami-${var.aws_region}"
"sourceAMI" = var.aws_ami
"sourceRegion" = var.aws_ami_region
},
local.tags,
)
}

6 changes: 3 additions & 3 deletions data/data/aws/master/main.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ resource "aws_network_interface" "master" {

tags = merge(
{
"Name" = "${var.cluster_id}-master-${count.index}"
"Name" = "${var.cluster_id}-master-${count.index}"
},
var.tags,
)
Expand Down Expand Up @@ -154,7 +154,7 @@ resource "aws_instance" "master" {

tags = merge(
{
"Name" = "${var.cluster_id}-master-${count.index}"
"Name" = "${var.cluster_id}-master-${count.index}"
},
var.tags,
)
Expand All @@ -169,7 +169,7 @@ resource "aws_instance" "master" {

volume_tags = merge(
{
"Name" = "${var.cluster_id}-master-${count.index}-vol"
"Name" = "${var.cluster_id}-master-${count.index}-vol"
},
var.tags,
)
Expand Down
52 changes: 37 additions & 15 deletions data/data/aws/variables-aws.tf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,39 @@ EOF
default = "1.0"
}

variable "custom_endpoints" {
type = map(string)

description = <<EOF
(optional) Custom AWS endpoints to override existing services.
Check - https://www.terraform.io/docs/providers/aws/guides/custom-service-endpoints.html

Example: `{ "key" = "value", "foo" = "bar" }`
EOF

default = {}
}

variable "aws_bootstrap_instance_type" {
type = string
type = string
description = "Instance type for the bootstrap node. Example: `m4.large`."
}

variable "aws_master_instance_type" {
type = string
type = string
description = "Instance type for the master node(s). Example: `m4.large`."
}

variable "aws_ami" {
type = string
type = string
description = "AMI for all nodes. An encrypted copy of this AMI will be used. Example: `ami-foobar123`."
}

variable "aws_ami_region" {
type = string
description = "Region for the AMI for all nodes. An encrypted copy of this AMI will be used. Example: `ami-foobar123`."
}

variable "aws_extra_tags" {
type = map(string)

Expand All @@ -35,12 +53,12 @@ EOF
}

variable "aws_master_root_volume_type" {
type = string
type = string
description = "The type of volume for the root block device of master nodes."
}

variable "aws_master_root_volume_size" {
type = string
type = string
Copy link
Contributor

@patrickdillon patrickdillon Apr 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like something weird is going on with tf-fmt, or I'm missing something... normally tf-fmt likes to have the = vertically aligned. I encountered something similar.

Copy link
Contributor Author

@abhinavdahiya abhinavdahiya Apr 17, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, i think we are using an old version of tf that had some weird bug... we should fix it and use a newer version..

description = "The size of the volume in gigabytes for the root block device of master nodes."
}

Expand Down Expand Up @@ -76,39 +94,43 @@ EOF
}

variable "aws_region" {
type = string
type = string
description = "The target AWS region for the cluster."
}

variable "aws_master_availability_zones" {
type = list(string)
type = list(string)
description = "The availability zones in which to create the masters. The length of this list must match master_count."
}

variable "aws_worker_availability_zones" {
type = list(string)
type = list(string)
description = "The availability zones to provision for workers. Worker instances are created by the machine-API operator, but this variable controls their supporting infrastructure (subnets, routing, etc.)."
}

variable "aws_vpc" {
type = string
default = null
type = string
default = null
description = "(optional) An existing network (VPC ID) into which the cluster should be installed."
}

variable "aws_public_subnets" {
type = list(string)
default = null
type = list(string)
default = null
description = "(optional) Existing public subnets into which the cluster should be installed."
}

variable "aws_private_subnets" {
type = list(string)
default = null
type = list(string)
default = null
description = "(optional) Existing private subnets into which the cluster should be installed."
}

variable "aws_publish_strategy" {
type = string
type = string
description = "The cluster publishing strategy, either Internal or External"
}
variable "aws_skip_region_validation" {
type = bool
description = "This decides if the AWS provider should validate if the region is known."
}
6 changes: 3 additions & 3 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,8 +115,8 @@ require (
gopkg.in/AlecAivazis/survey.v1 v1.8.9-0.20200217094205-6773bdf39b7f
gopkg.in/ini.v1 v1.51.0
gopkg.in/yaml.v2 v2.2.8
k8s.io/api v0.17.2
k8s.io/apimachinery v0.17.3
k8s.io/api v0.18.0
k8s.io/apimachinery v0.18.0
k8s.io/client-go v12.0.0+incompatible
k8s.io/klog v1.0.0
k8s.io/utils v0.0.0-20191217005138-9e5e9d854fcc
Expand All @@ -133,7 +133,7 @@ replace (
github.com/hashicorp/terraform-plugin-sdk => github.com/openshift/hashicorp-terraform-plugin-sdk v1.6.0-openshift // Pin to fork with public rpc types
github.com/metal3-io/baremetal-operator => github.com/openshift/baremetal-operator v0.0.0-20200206190020-71b826cc0f0a // Use OpenShift fork
github.com/metal3-io/cluster-api-provider-baremetal => github.com/openshift/cluster-api-provider-baremetal v0.0.0-20190821174549-a2a477909c1d // Pin OpenShift fork
github.com/openshift/api => github.com/openshift/api v0.0.0-20200210091934-a0e53e94816b // Pin API
github.com/openshift/api => github.com/openshift/api v0.0.0-20200413201024-c6e8c9b6eb9a // Pin API
github.com/openshift/machine-config-operator => github.com/openshift/machine-config-operator v0.0.1-0.20200130220348-e5685c0cf530 // Pin MCO so it doesn't get downgraded
github.com/terraform-providers/terraform-provider-azurerm => github.com/openshift/terraform-provider-azurerm v1.41.1-openshift-3 // Pin to openshift fork with IPv6 fixes
google.golang.org/api => google.golang.org/api v0.13.0 // Pin to version required by tf-provider-google
Expand Down
Loading