CI/CD pipeline examples: GCP, AWS and Azure infra stacks in GitLab, ArgoCD, Jenkins

[prasek]

Provide working examples and associated blog posts describing how to integrate Crossplane into various CI/CD pipelines for continuous deployment of Kubernetes apps and the cloud services they depend on.

Part of the 0.3 release

Why? What problem are we solving?

Crossplane simplifies k8s app deployments into different target environments (dev, test, prod) that often use their own networks, subnets, k8s clusters, cloud services, credentials, and security configuration.

For shared environments supporting multiple app teams, it can save a lot of time & work to configure each target environment once (networks, subnets, target k8s clusters, cloud service provisioners, cloud credentials) esp. since it can take several minutes to provision k8s clusters on demand and esp. in dev/test environments the k8s apps & cloud service instances are often ephemeral and transient in nature.

Even when an app team owns their own target environments (dev/test/prod) and k8s clusters, there are often multiple developers on a team sharing a (dev/test) cluster, and k8s app deployments (with cloud services) are frequently torn down (from-scratch vs. in-place upgrades) and the deployed k8s app instances need to be rapidly cycled as new commits land on master or changes are promoted.

Even if you're the only developer on an app and you have your own target environments (dev/test/prod) that you exclusively own and manage you probably don't want the time hit of k8s target cluster creation every time you cycle your k8s app deployment with cloud services in dev/test. So even then configuring your target environments once and rapidly cycling your k8s app instances is desirable.

So there are 2 primary problems:
(1) config of target environments (networks, subnets, target k8s clusters, ...), to support
(2) rapid deploy & reclaim k8s apps and cloud service dependencies in dev/test and maybe prod.

It's often considered a best practice to have these configurations stored 'as code' in a VCS, and then have changes committed trigger a traditional CI/CD pipeline like Jenkins or GitLab or have a GitOps style flow that polls Git and synchronizes desired state (e.g. WeaveWorks Flux, Google Anthos, etc).

So as a Crossplane user:

• How should I use Crossplane infra stacks and app stacks from my CI/CD pipeline of choice?
• Is there an example I can use to get started quickly?
• Should I use kubectl apply or are is there a CLI tool I should use instead (e.g. to build, publish, and then apply stacks)?

What does it look like when we're done?

For (Jenkins pipeline, GitLab pipeline, GitOps flow)x(GCP, AWS, Azure stacks):

• Blog describing how Crossplane should be used and working examples I can copy to:
  • configure target environments:
    • networks, subnets, target k8s clusters, default resource classes, credentials, etc.
  • rapidly cycle k8s app instances with cloud services
    • no app stacks, using resource claims only
    • with app stacks, pushing new app stack version
      • upgrading (in-place)
      • redeploying app instance (from-scratch)
• Split into separate epic? Dogfood repo/pipeline is set up and running to implement a true GitOps continuous delivery scenario. Some infrastructure and application is deployed and managed entirely from this GitOps pipeline. Mattermost. config repo. pipelines all setup in that repo. first time pipeline runs it spins up Mattermost, which exists, then change to repo, edit some config, pipeline runs and updates the real matter most server in the wild. Live up-and-running instance controlled and configured by GitOps pipeline and repo.
• Crossplane enhancements to improve CI/CD pipeline UX as required

How could this be demonstrated?

1. Crossplane user reads the blog post describing this new functionality
2. Click through to the updated user guide
3. Try out the updated example(s)
4. View the dogfood GitOps CD pipeline as a real functional example

For (Jenkins pipeline, GitLab pipeline, GitOps flow)x(GCP stack, AWS stack, Azure stack):

• Read blog, follow examples, demonstrate using Crossplane infra & apps stacks

Releases:

• v0.4.0
  • 887 - 890 - change controller runtime to use resolveDep; for each stack add new type for each type being referenced. not too much work required.
  • Update crossplane.io services & stacks guides with new kubectl apply -f techniques and repos #905 update services guides and stacks setup guides to reflect kubectl apply -f technique, sample infra & app repos
• v0.5.0
  • continuous deployment examples for GitLab, ArgoCD, Jenkins

Related:

• One pager for Cross Resource Referencing Inter-resource attribute references #707, Cross Resource Referencing design doc #874

• Implement Cross Resource Referencing

  • Crossplane runtime Implement Cross Resource Referencing in Crossplane runtime #887
  • Crossplane core Crossplane Core: Implement Cross Resource Referencing for resource types #948
  • GCP resource types GCP: Implement Cross Resource Referencing for resource types #888
  • AWS resource types AWS: Implement Cross Resource Referencing for resource types #889
  • Azure resource types Azure: Implement Cross Resource Referencing for resource types  #890
  • Update crossplane.io services guides and stacks guides to reflect kubectl apply -f technique, w/ working sample infra and app repos - Update crossplane.io services & stacks guides with new kubectl apply -f techniques and repos #905

• Write a user guide to use a gitops CI/CD system for Crossplane infra and app repositories

  • GitLab GitLab: Write a user guide to use a gitops CI/CD system for Crossplane infra and app repositories, for the 3 cloud providers #886
    • GCP (ETA Oct 11)
    • AWS
    • Azure
  • ArgoCD ArgoCD: Write a user guide to use a gitops CI/CD system for Crossplane infra and app repositories, for the 3 cloud providers #904
    • GCP
    • AWS
    • Azure
  • Jenkin Jenkins: Write a user guide to use a gitops CI/CD system for Crossplane infra and app repositories, for the 3 cloud providers #902
    • GCP
    • AWS
    • Azure

• Good to have:

  • Static provisioning by app developers
  • Immutable fields Immutable resource fields should be taken into account by an admission webhook #727

[muvaf]

@negz Why is this not considered useful anymore?