GCP infra stack: single-region secure connectivity for GKE & CloudSQL, CloudMemorystore, Buckets

[negz]

GCP infra stack users can configure everything for their environment from the Kubernetes API including networking, subnets, and secure connectivity between app deployments in a target cluster and the dynamically provisioned cloud services they depend on.

Part of the 0.3 release

Why? What problem are we solving?

A Stack is a set of CRDs and controllers that extend Crossplane. An infrastructure stack adds support for infrastructure (e.g. databases, VMs, etc), as opposed to adding support for an application (e.g. Wordpress, GitLab).

We'd like to take our first steps in this direction by breaking Crossplane's current Google Cloud Platform (GCP) managed resource functionality out into a GCP stack (covered by #612). Once the stack is broken out, we want to ensure it:

• Could be used by an application stack (such as a Wordpress stack) to deploy to GCP, including allowing cluster administrators to configure default resource classes that ensure network connectivity per Add a draft one pager for MVP resource connectivity #606.
• Establishes a solid foundation for the community to build new GCP related controllers.

What does it look like when we're done?

• The GCP stack contains v1alpha2 variants of all existing v1alpha1 GCP managed resources.
• The GCP stack contains v1alpha2 variants of all existing v1alpha1 GCP managed resource classes. This is predicated on the work tracked in Resource classes can be validated and annotated #613.
• The GCP stack is based on the latest Crossplane best practices:
  • The stack is built on kubebuilder 0.2.0 (possibly a beta thereof), including controller-tools, and the controller-runtime.
  • All managed resources are implemented using the nascent (Implement Generic Managed Reconciler #603) generic managed resource reconciler.
  • All resource claims are implemented using the generic resource claim reconciler.
• The GCP stack contains MVP resource connectivity between GKECluster and CloudSQLInstance managed resources as described in Add a draft one pager for MVP resource connectivity #606. This implies
  • Adding CRDs and controllers for the Network and Subnetwork managed resources under the vpc.gcp.crossplane.io/v1alpha1 API group.
  • Ensuring 'high fidelity' (as described in Add a draft one pager for MVP resource connectivity #606) connectivity related managed resource spec fields.
• An application owner can leverage the GCP infrastructure stack when deploying an app stack. e.g. A KubernetesApplication can be scheduled and deployed to a GCP backed KubernetesCluster, and leverage GCP backed resource claims.

How could this be demonstrated?

1. Crossplane user reads the blog post describing this new functionality
2. Click through to the updated user guide
3. Try out the updated example(s)

For example:

1. Begin with 'core' Crossplane, i.e. just the extension manager without any GCP support.
2. Install the GCP infrastructure stack.
3. Create Network and Subnetwork managed resources.
4. Create a CloudSQLInstanceClass that will use the aforementioned Network as the whitelisted private network.
5. Create a GKEClusterClass that will create nodes in the aforementioned Network and Subnetwork.
6. Create MySQLInstance and KubernetesCluster resource claims referencing the aforementioned resource classes.
7. Deploy wordpress to the aforementioned KubernetesCluster. Ensure it can access the aforementioned MySQLInstance.

Related

• Assumes Resource classes can be validated and annotated #613 handles: Create strongly typed resource classes for each managed kind
• GCP: Author security/connectivity design doc with GCP specifics
• GCP: Define new networking types and controllers
  • Network CRD and controller with high fidelity GCP: Support for Network Resource with High Fidelity #654
  • Subnetwork CRD and controller with high fidelity GCP: Support for Subnetwork Resource with High Fidelity #657
• GCP: Generic managed reconciler for all existing GCP resources (for v1alpha2) GCP: Refactor all managed resource controllers into generic managed reconciler crossplane-contrib/provider-gcp#3
  • Depends on Converge on a consistent controller design #269 Converge on a consistent controller design
  • Version bump of GCP CRDs -> v1alpha2
• GCP: Validate e2e portable wordpress stack [Publish a portable wordpress app stack to a registry #572 ] deployed securely & accessible GCP: Wordpress Example with New Network Resources #661
  • Connected Wordpress demo e2e
  • GCP: Update examples and README files containing Wordpress GCP examples handled as part of Publish a portable wordpress app stack to a registry #572
• GCP: Update API status table for infra stack capabilities, API version level, etc.

[muvaf]

I think we can create the sub-issues as individual tasks in the following form:

• Version bump of GCP CRDs -> v1alpha2
• Create strongly typed resource classes for each managed kind
• Generic managed reconciler for all existing GCP resources (for v1alpha2)
• Network CRD and controller with high fidelity
• Subnetwork CRD and controller with high fidelity
• Wordpress app stack YAMLs that use v1alpha2 managed resources and network CRDs
• Connected Wordpress demo e2e
• Update examples and README files containing Wordpress GCP examples.

[hasheddan]

It might be worth including the creation of strongly typed resource classes for each managed kind in this process, as it will also require a minor update to each claim controller.

[muvaf]

@hasheddan just added, thanks! I'll move the bulletpoints up into the first issue description once we finalize and I create the issues.