DO NOT MERGE: Prototype GCP installer-provisioned-infra #1109

smarterclayton · 2019-01-22T17:34:19Z

This is a prototype to assist in documenting how someone would add a new platform to OpenShift.

STATUS: Currently blocked because MCO panics with no such platform

This wires a minimum of create logic and behavior to enable a new platform 'google' (name should be 'gcp', but terraform fails when platform name != terraform-cloud-provider name). The destroy function is wired directly to terraform for now and is considered a bare minimum step.

The terraform steps are consistent with AWS and the 3.11 provisioning playbooks in openshift-ansible - roughly:

Create bootstrap in zone[1], masters in zones[1..3]
Use internal network load balancer for masters
Use TCP proxy for external load balancer
(no domain name setup yet)
Directly inject bootstrap.ign to user-data since we can on GCP
Using a hardcoded maipo image since we don't have a continuous run
Create a service account for the bootstrap
Wire through instance size, disk, and disk size

TODO:

create a document to describe the process and how we would selectively enable it, as well as set up CI (WIP: doc: Begin a document on adding a new OpenShift platform #1112)
domain names
create IAM profile and assign to service account
add cloud credentials and survey questions to get credentials
MCO enablement
???

This wires a minimum of create logic and behavior to enable a new platform 'google' (name should be 'gcp', but terraform fails when platform name != terraform-cloud-provider name). The destroy function is wired directly to terraform for now and is considered a bare minimum step. The terraform steps are consistent with AWS and the 3.11 provisioning playbooks in openshift-ansible - roughly: 1. Create bootstrap in zone[1], masters in zones[1..3] 2. Use internal network load balancer for masters 3. Use TCP proxy for external load balancer 4. (no domain name setup yet) 5. Directly inject bootstrap.ign to user-data since we can on GCP 6. Using a hardcoded maipo image since we don't have a continuous run 7. Create a service account for the bootstrap 8. Wire through instance size, disk, and disk size

openshift-ci-robot · 2019-01-22T17:34:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [smarterclayton]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

russellb · 2019-01-22T19:03:43Z

@flaper87 Didn't you write some docs tracking everything that was needed for OpenStack that could possibly help with the documentation goal written here?

smarterclayton · 2019-01-22T19:08:44Z

please comment on #1112

cgwalters · 2019-01-22T20:10:29Z

pkg/asset/installconfig/google/google.go

+	err := survey.Ask([]*survey.Question{
+		{
+			Prompt: &survey.Input{
+				Message: "AWS Access Key ID",


Some AWS leftovers here.

yeah, I don't take credentials yet. Once we settle on behavior of the various operators on unrecognized clouds I'm going to go open a bunch of PRs /bugs and then push through more.

seanmalloy · 2019-01-24T05:12:10Z

data/data/google/bootstrap/main.tf

+    user-data = "${var.ignition}"
+  }
+
+  tags = ["ocp", "ocp-master"]


I recommend adding the the ability for a user to add additional tags to this list. This will be necessary for environments with restrictive GCP firewalls that require specific tags to be set to allow network communication.

seanmalloy · 2019-01-24T05:14:53Z

data/data/google/bootstrap/main.tf

+
+  network_interface = {
+    network    = "${var.subnetwork != "" ? "" : var.network}"
+    subnetwork = "${var.subnetwork}"


Some environments may require using a GCP Shared VPC. The above code will not work with a Shared VPC.

seanmalloy · 2019-01-24T05:32:15Z

data/data/google/master/main.tf

@@ -0,0 +1,75 @@
+resource "google_compute_instance" "master" {


You might want to consider adding a scheduling block to the master GCE instance. Something like this ...

scheduling { automatic_restart = "true" on_host_maintenance = "MIGRATE" preemptible = "false" }

By default the Terraform Google Cloud Provider will not enable automatic_restart if you don't explicitly set it. Looks like this will be be fixed in version 2.0 of the provider. See hashicorp/terraform-provider-google#2638 for details.

seanmalloy · 2019-01-24T05:34:18Z

data/data/google/master/main.tf

+
+  network_interface = {
+    network    = "${var.subnetwork != "" ? "" : var.network}"
+    subnetwork = "${var.subnetwork}"


Similar comment as above regarding GCP Shared VPC. Some users may require using a Shared VPC.

seanmalloy · 2019-01-24T05:55:48Z

data/data/google/network/master-lb.tf

+  health_checks = ["${google_compute_health_check.masters.self_link}"]
+}
+
+resource "google_compute_target_tcp_proxy" "master" {


It looks like this load balancer is Internet facing. Do you have plans to support a GCP ILB at some point in the future?

https://cloud.google.com/load-balancing/docs/internal/

seanmalloy · 2019-01-24T06:13:03Z

pkg/asset/installconfig/google/google.go

@@ -0,0 +1,163 @@
+// Package aws collects AWS-specific configuration.


Maybe change this to // Package google collects GCP-specfic configuration.

seanmalloy · 2019-01-24T06:26:34Z

pkg/destroy/google/google.go

+	Logger   logrus.FieldLogger
+}
+
+// New returns an OpenStack destroyer from ClusterMetadata.


s/OpenStack/GCP/

seanmalloy · 2019-01-24T06:36:43Z

pkg/types/google/validation/platform.go

+)
+
+var (
+	// Regions is a map of the known AWS regions. The key of the map is


seanmalloy · 2019-01-24T06:37:37Z

pkg/types/google/validation/platform.go

+	// the short name of the region. The value of the map is the long
+	// name of the region.
+	Regions = map[string]string{
+		"us-central1": "Ohio",


Probably doesn't matter, but us-central1 is in Iowa.

openshift-ci-robot · 2019-02-04T05:52:59Z

@smarterclayton: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2019-05-15T19:49:39Z

@smarterclayton: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
ci/prow/tf-fmt	`3c4e92e`	link	`/test tf-fmt`
ci/prow/golint	`3c4e92e`	link	`/test golint`
ci/prow/e2e-aws-upgrade	`3c4e92e`	link	`/test e2e-aws-upgrade`
ci/prow/verify-vendor	`3c4e92e`	link	`/test verify-vendor`
ci/prow/e2e-aws	`3c4e92e`	link	`/test e2e-aws`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

abhinavdahiya · 2019-07-24T22:15:10Z

IPI for GCP was implemented using this. and this is now obsolete.

/close

openshift-ci-robot · 2019-07-24T22:15:12Z

@abhinavdahiya: Closed this PR.

Details

In response to this:

IPI for GCP was implemented using this. and this is now obsolete.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

smarterclayton added 2 commits January 21, 2019 00:34

vendor: Add terraform-google provider

0b38679

openshift-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jan 22, 2019

openshift-ci-robot requested review from russellb and steveej January 22, 2019 17:34

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 22, 2019

cgwalters reviewed Jan 22, 2019

View reviewed changes

smarterclayton mentioned this pull request Jan 23, 2019

mco: If the platform is unrecognized, treat the platform as 'none' openshift/machine-config-operator#340

Merged

seanmalloy reviewed Jan 24, 2019

View reviewed changes

pkg/destroy/google/google.go

Logger logrus.FieldLogger

}

// New returns an OpenStack destroyer from ClusterMetadata.

Copy link

seanmalloy Jan 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/OpenStack/GCP/

seanmalloy reviewed Jan 24, 2019

View reviewed changes

pkg/types/google/validation/platform.go

)

var (

// Regions is a map of the known AWS regions. The key of the map is

Copy link

seanmalloy Jan 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/AWS/GCP/

seanmalloy reviewed Jan 24, 2019

View reviewed changes

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 4, 2019

openshift-ci-robot closed this Jul 24, 2019

		@@ -0,0 +1,75 @@
		resource "google_compute_instance" "master" {

		@@ -0,0 +1,163 @@
		// Package aws collects AWS-specific configuration.

DO NOT MERGE: Prototype GCP installer-provisioned-infra #1109

DO NOT MERGE: Prototype GCP installer-provisioned-infra #1109

Uh oh!

Conversation

smarterclayton commented Jan 22, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented Jan 22, 2019

Uh oh!

russellb commented Jan 22, 2019

Uh oh!

smarterclayton commented Jan 22, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Feb 4, 2019

Uh oh!

openshift-ci-robot commented May 15, 2019

Uh oh!

abhinavdahiya commented Jul 24, 2019

Uh oh!

openshift-ci-robot commented Jul 24, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

smarterclayton commented Jan 22, 2019 •

edited

Loading