zenml-io · bcdurak · Jul 9, 2024 · Jun 28, 2024 · Jun 28, 2024 · Jun 28, 2024
diff --git a/README.md b/README.md
@@ -133,9 +133,46 @@ if __name__ == "__main__":
 
 ![Running a ZenML pipeline](/docs/book/.gitbook/assets/readme_basic_pipeline.gif)
 
-### Deploy workloads easily on your production infrastructure
+### Provision a MLOps stack or reuse existing infrastructure easily
 
-The framework is a gentle entry point for practitioners to build complex ML pipelines with little knowledge required of the underlying infrastructure complexity. ZenML pipelines can be run on AWS, GCP, Azure, Airflow, Kubeflow and even on Kubernetes without having to change any code or know underlying internals.
+The framework is a gentle entry point for practitioners to build complex ML pipelines with little knowledge required of the underlying infrastructure complexity. ZenML pipelines can be run on AWS, GCP, Azure, Airflow, Kubeflow and even on Kubernetes without having to change any code or know underlying internals. 
+
+There are public templates to easily provision a simple MLOps stack that can be used with ZenML:
+
+<div align="center">
+
+[![Create a AWS MLOps Stack](https://img.shields.io/badge/AWS-%23FF9900.svg?style=for-the-badge&logo=amazon-aws&logoColor=white)](https://console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/create/review?stackName=zenml-stack&templateURL=https://zenml-cf-templates.s3.eu-central-1.amazonaws.com/aws-ecr-s3-sagemaker.yaml)
+
+[![Create a GCP MLOps Stack](https://img.shields.io/badge/GoogleCloud-%234285F4.svg?style=for-the-badge&logo=google-cloud&logoColor=white)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https://github.com/zenml-io/zenml&cloudshell_working_dir=infra&cloudshell_open_in_editor=gcp-gar-gcs-vertex.yaml,gcp-gar-gcs-vertex-config.yaml&cloudshell_print=gcp-gar-gcs-vertex.txt&cloudshell_git_branch=feature/prd-482-one-click-stacks)
+
+[![Create an Azure MLOps Stack](https://img.shields.io/badge/azure-%230072C6.svg?style=for-the-badge&logo=microsoftazure&logoColor=white)](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https://github.com/zenml-io/zenml&cloudshell_working_dir=infra&cloudshell_open_in_editor=gcp-gar-gcs-vertex.yaml,gcp-gar-gcs-vertex-config.yaml&cloudshell_print=gcp-gar-gcs-vertex.txt&cloudshell_git_branch=feature/prd-482-one-click-stacks)
+
+</div>
+
+Or, use a simple CLI command:
+
+```bash
+zenml stack deploy --provider aws  # can be gcp, azure
+```
+
+Already have existing resources? No problem, you can register them easily:
+
+```bash
+zenml stack register --provider aws  # can be gcp, azure
+```
+
+TODO: Might be nice to have a GIF here of zenml stack register.
+
+Read more about [ZenML stacks](https://docs.zenml.io/user-guide/production-guide/understand-stacks).
+
+### Run workloads easily on your production infrastructure
+
+Once you have your MLOps stack configured, you can easily run workloads on it:
+
+```bash
+zenml stack set k8s  # Set a stack with kubernetes orchestrator
+python run.py
+```
 
 ```python
 from zenml.config import ResourceSettings, DockerSettings
@@ -150,14 +187,8 @@ def training(...):
 	...
 ```
 
-```bash
-zenml stack set k8s  # Set a stack with kubernetes orchestrator
-python run.py
-```
-
 ![Workloads with ZenML](/docs/book/.gitbook/assets/readme_compute.gif)
 
-
 ### Track models, pipeline, and artifacts
 
 Create a complete lineage of who, where, and what data and models are produced.

diff --git a/docs/book/component-guide/artifact-stores/gcp.md b/docs/book/component-guide/artifact-stores/gcp.md
@@ -21,10 +21,6 @@ You should use the GCS Artifact Store when you decide to keep your ZenML artifac
 
 ### How do you deploy it?
 
-{% hint style="warning" %}
-The GCP artifact store (and GCP integration in general) currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
-{% endhint %}
-
 The GCS Artifact Store flavor is provided by the GCP ZenML integration, you need to install it on your local machine to be able to register a GCS Artifact Store and add it to your stack:
 
 ```shell

diff --git a/docs/book/component-guide/artifact-stores/s3.md b/docs/book/component-guide/artifact-stores/s3.md
@@ -21,6 +21,13 @@ You should use the S3 Artifact Store when you decide to keep your ZenML artifact
 
 ### How do you deploy it?
 
+{% hint style="info" %}
+Don't want to deploy the artifact store manually? Check out the
+[easy cloud deployment wizard](../../how-to/stack-deployment/deploy-a-cloud-stack.md)
+or the [easy cloud registration wizard](../../how-to/stack-deployment/register-a-cloud-stack.md)
+for a shortcut on how to deploy & register this stack component.
+{% endhint %}
+
 The S3 Artifact Store flavor is provided by the S3 ZenML integration, you need to install it on your local machine to be able to register an S3 Artifact Store and add it to your stack:
 
 ```shell

diff --git a/docs/book/component-guide/container-registries/aws.md b/docs/book/component-guide/container-registries/aws.md
@@ -15,6 +15,13 @@ You should use the AWS container registry if:
 
 ### How to deploy it
 
+{% hint style="info" %}
+Don't want to deploy the container registry manually? Check out the
+[easy cloud deployment wizard](../../how-to/stack-deployment/deploy-a-cloud-stack.md)
+or the [easy cloud registration wizard](../../how-to/stack-deployment/register-a-cloud-stack.md)
+for a shortcut on how to deploy & register this stack component.
+{% endhint %}
+
 The ECR registry is automatically activated once you create an AWS account. However, you'll need to create a `Repository` in order to push container images to it:
 
 * Go to the [ECR website](https://console.aws.amazon.com/ecr).

diff --git a/docs/book/component-guide/container-registries/gcp.md b/docs/book/component-guide/container-registries/gcp.md
@@ -19,10 +19,6 @@ You should use the GCP container registry if:
 
 ### How to deploy it
 
-{% hint style="warning" %}
-The GCP container registry (and GCP integration in general) currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
-{% endhint %}
-
 When using the Google Artifact Registry, you need to:
 
 * enable it [here](https://console.cloud.google.com/marketplace/product/google/artifactregistry.googleapis.com)
@@ -36,7 +32,7 @@ A GCP Container Registry can be deployed directly from the ZenML CLI:
 zenml container-registry deploy gcp_container_registry --flavor=gcp --provider=gcp ...
 ```
 
-You can pass other configurations specific to the stack components as key-value arguments. If you don't provide a name, a random one is generated for you. For more information about how to work use the CLI for this, please refer to the [dedicated documentation section](../../how-to/stack-deployment/deploy-a-stack-component.md).
+You can pass other configurations specific to the stack components as key-value arguments. If you don't provide a name, a random one is generated for you. For more information about how to work use the CLI for this, please refer to the [dedicated documentation section](../../how-to/stack-deployment/deploy-a-stack-using-mlstacks.md).
 
 ## How to find the registry URI
 

diff --git a/docs/book/component-guide/image-builders/gcp.md b/docs/book/component-guide/image-builders/gcp.md
@@ -20,10 +20,6 @@ In order to use the ZenML Google Cloud image builder you need to enable Google C
 
 ### How to use it
 
-{% hint style="warning" %}
-The GCP image builder (and GCP integration in general) currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
-{% endhint %}
-
 To use the Google Cloud image builder, we need:
 
 *   The ZenML `gcp` integration installed. If you haven't done so, run:

diff --git a/docs/book/component-guide/orchestrators/sagemaker.md b/docs/book/component-guide/orchestrators/sagemaker.md
@@ -26,6 +26,13 @@ The ZenML Sagemaker orchestrator works with [Sagemaker Pipelines](https://aws.am
 
 ## How to deploy it
 
+{% hint style="info" %}
+Don't want to deploy the orchestrator manually? Check out the
+[easy cloud deployment wizard](../../how-to/stack-deployment/deploy-a-cloud-stack.md)
+or the [easy cloud registration wizard](../../how-to/stack-deployment/register-a-cloud-stack.md)
+for a shortcut on how to deploy & register this stack component.
+{% endhint %}
+
 In order to use a Sagemaker AI orchestrator, you need to first deploy [ZenML to the cloud](../../getting-started/deploying-zenml/README.md). It would be recommended to deploy ZenML in the same region as you plan on using for Sagemaker, but it is not necessary to do so. You must ensure that you are connected to the remote ZenML server before using this stack component.
 
 The only other thing necessary to use the ZenML Sagemaker orchestrator is enabling the relevant permissions for your particular role.

diff --git a/docs/book/component-guide/orchestrators/vertex.md b/docs/book/component-guide/orchestrators/vertex.md
@@ -30,10 +30,6 @@ In order to quickly enable APIs, and create other resources necessary for using
 
 ## How to use it
 
-{% hint style="warning" %}
-The Vertex Orchestrator (and GCP integration in general) currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
-{% endhint %}
-
 To use the Vertex orchestrator, we need:
 
 *   The ZenML `gcp` integration installed. If you haven't done so, run

diff --git a/docs/book/component-guide/step-operators/vertex.md b/docs/book/component-guide/step-operators/vertex.md
@@ -20,10 +20,6 @@ You should use the Vertex step operator if:
 
 ### How to use it
 
-{% hint style="warning" %}
-The GCP step operator (and GCP integration in general) currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
-{% endhint %}
-
 To use the Vertex step operator, we need:
 
 *   The ZenML `gcp` integration installed. If you haven't done so, run

diff --git a/docs/book/how-to/popular-integrations/aws-guide.md b/docs/book/how-to/popular-integrations/aws-guide.md
@@ -6,6 +6,19 @@ description: A simple guide to create an AWS stack to run your ZenML pipelines
 
 This page aims to quickly set up a minimal production stack on AWS. With just a few simple steps, you will set up an IAM role with specifically-scoped permissions that ZenML can use to authenticate with the relevant AWS resources.
 
+{% hint style="info" %}
+**Don't want to learn how to deploy and register stack components manually? Feel free to skip this chapter
+and read [how to easily register a cloud stack](../../how-to/stack-deployment/register-a-cloud-stack.md)
+if you have existing infrastructure, or read [how to deploy a cloud stack in one click](../../how-to/stack-deployment/deploy-a-cloud-stack.md).**
+
+Or simply try running one of:
+
+```shell
+zenml stack register --provider aws
+zenml stack deploy --provider aws
+```
+{% endhint %}
+
 ## 1) Set up credentials and local environment
 
 To follow this guide, you need:

diff --git a/docs/book/how-to/popular-integrations/gcp-guide.md b/docs/book/how-to/popular-integrations/gcp-guide.md
@@ -4,8 +4,17 @@ description: A simple guide to quickly set up a minimal stack on GCP.
 
 # Set up a minimal GCP stack
 
-{% hint style="warning" %}
-The GCP integration currently only works for Python versions <3.11. The ZenML team is aware of this dependency clash/issue and is working on a fix. For now, please use Python <3.11 together with the GCP integration.
+{% hint style="info" %}
+**Don't want to learn how to deploy and register stack components manually? Feel free to skip this chapter
+and read [how to easily register a cloud stack](../../how-to/stack-deployment/register-a-cloud-stack.md)
+if you have existing infrastructure, or read [how to deploy a cloud stack in one click](../../how-to/stack-deployment/deploy-a-cloud-stack.md).**
+
+Or simply try running one of:
+
+```shell
+zenml stack register --provider gcp
+zenml stack deploy --provider gcp
+```
 {% endhint %}
 
 This page aims to quickly set up a minimal production stack on GCP. With just a few simple steps you will set up a service account with specifically-scoped permissions that ZenML can use to authenticate with the relevant GCP resources.

diff --git a/docs/book/how-to/stack-deployment/README.md b/docs/book/how-to/stack-deployment/README.md
@@ -1,16 +1,12 @@
 ---
-description: Deploying your stack components directly from the ZenML CLI
+description: Stacks are the configuration of your infrastructure.
 ---
 
-# ⚒️ Manage stacks
+# Managing stacks & components
 
-The first step in running your pipelines on remote infrastructure is to deploy all the components that you would need, like an MLflow tracking server, a Seldon Core model deployer, and more to your cloud.
+The concept of a [stack](../../user-guide/production-guide/understand-stacks.md) is a fundamental component of the ZenML framework. Put simply, a stack is the configuration of the infrastructure and tooling that defines where and how a pipeline executes. Let’s take a simple example. Here is a simple pipeline in ZenML:
 
-This can bring plenty of benefits like scalability, reliability, and collaboration. ZenML eases the path to production by providing a seamless way for all tools to interact with others through the use of abstractions. However, one of the most painful parts of this process, from what we see on our Slack and in general, is the deployment of these stack components.
-
-## Deploying and managing MLOps tools is tricky 😭😵‍💫
-
-It is not trivial to set up all the different tools that you might need for your pipeline.
+However, deploying and managing a MLOps stack is tricky 😭😵‍💫. It is not trivial to set up all the different tools that you might need for your pipeline.
 
 * 🌈 Each tool comes with a certain set of requirements. For example, a Kubeflow installation will require you to have a Kubernetes cluster, and so would a Seldon Core deployment.
 * 🤔 Figuring out the defaults for infra parameters is not easy. Even if you have identified the backing infra that you need for a stack component, setting up reasonable defaults for parameters like instance size, CPU, memory, etc., needs a lot of experimentation to figure out.
@@ -21,41 +17,48 @@ It is not trivial to set up all the different tools that you might need for your
 
 All of these points make taking your pipelines to production a more difficult task than it should be. We believe that the expertise in setting up these often-complex stacks shouldn't be a prerequisite to running your ML pipelines.
 
-Thus, to make even this process easier for our users, we have created the `deploy` CLI which allows you to quickly get started with a full-fledged MLOps stack using only a few commands. You can choose to deploy individual stack components through the stack-component CLI or deploy a stack with multiple components together (a tad more manual steps).
-
-## What is `mlstacks`?
-
-[MLStacks](https://mlstacks.zenml.io) is a [Python package](https://pypi.org/project/mlstacks/) that allows you to quickly spin up MLOps infrastructure using Terraform. It is designed to be used with [ZenML](https://zenml.io), but can be used with any MLOps tool or platform.
-
-The ZenML CLI has special subcommands that allow you to deploy individual stack components as well as whole stacks using MLStacks. These stacks will be useful for you if:
-
-* You are at the start of your MLOps journey, and would like to explore different tools.
-* You are looking for guidelines for production-grade deployments.
-
-## Installing the mlstacks extra
-
-To install `mlstacks`, either run `pip install mlstacks` or `pip install "zenml[mlstacks]"` to install it along with ZenML.
-
-MLStacks uses Terraform on the backend to manage infrastructure. You will need to have Terraform installed. Please visit [the Terraform docs](https://learn.hashicorp.com/tutorials/terraform/install-cli#install-terraform) for installation instructions.
-
-MLStacks also uses Helm to deploy Kubernetes resources. You will need to have Helm installed. Please visit [the Helm docs](https://helm.sh/docs/intro/install/#from-script) for installation instructions.
-
-## Deploying a stack component
-
-The ZenML CLI allows you to deploy individual stack components using the `deploy` subcommand which is implemented for all supported stack components. You can find the list of supported stack components [here](../stack-deployment/README.md).
-
-## Deploying a stack
-
-For deploying a full stack, use the `zenml stack deploy` command. See the [stack deployment](deploy-a-stack-using-mlstacks.md) page for more details of which cloud providers and stack components are supported.
-
-## How does `mlstacks` work?
-
-MLStacks is built around the concept of a stack specification. A stack specification is a YAML file that describes the stack and includes references to component specification files. A component specification is a YAML file that describes a component. (Currently all deployments of components (in various combinations) must be defined within the context of a stack.)
-
-ZenML handles the creation of stack specifications for you when you run one of the `deploy` subcommands using the CLI. A valid specification is generated and used by `mlstacks` to deploy your stack using Terraform. The Terraform definitions and state are stored in your global configuration directory along with any state files generated while deploying your stack.
-
-Your configuration directory could be in a number of different places depending on your operating system, but read more about it in the [Click docs](https://click.palletsprojects.com/en/8.1.x/api/#click.get\_app\_dir) to see which location applies to your situation.
-
-<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><mark style="color:purple;"><strong>Deploy stack components individually</strong></mark></td><td>Individually deploying different stack components.</td><td><a href="deploy-a-stack-component.md">deploy-a-stack-component.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Deploy a stack with multiple components using mlstacks</strong></mark></td><td>Deploying an entire stack with ZenML's `mlstacks` package.</td><td><a href="../../how-to/stack-deployment/deploy-a-stack-using-mlstacks.md">../../how-to/stack-deployment/deploy-a-stack-using-mlstacks.md</a></td></tr><tr><td><mark style="color:purple;"><strong>Contribute new components or flavors</strong></mark></td><td>Creating your custom stack component solutions.</td><td><a href="../../../../CONTRIBUTING.md">CONTRIBUTING.md</a></td></tr></tbody></table>
+This docs section consists of information that makes it easier to provision, configure, and extend stacks and components in ZenML.
+
+<table data-view="cards">
+  <thead>
+    <tr>
+      <th></th>
+      <th></th>
+      <th data-hidden data-card-target data-type="content-ref"></th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><mark style="color:purple;"><strong>Deploy a cloud stack</strong></mark></td>
+      <td>Description of deploying a cloud stack.</td>
+      <td><a href="./deploy-a-cloud-stack.md">./deploy-a-cloud-stack.md</a></td>
+    </tr>
+    <tr>
+      <td><mark style="color:purple;"><strong>Register a cloud stack</strong></mark></td>
+      <td>Description of registering a cloud stack.</td>
+      <td><a href="./register-a-cloud-stack.md">./register-a-cloud-stack.md</a></td>
+    </tr>
+    <tr>
+      <td><mark style="color:purple;"><strong>Deploy stack/components using mlstacks</strong></mark></td>
+      <td>Deploying an entire stack with ZenML's `mlstacks` package.</td>
+      <td><a href="./deploy-a-stack-using-mlstacks.md">./deploy-a-stack-using-mlstacks.md</a></td>
+    </tr>
+    <tr>
+      <td><mark style="color:purple;"><strong>Reference secrets in stack configuration</strong></mark></td>
+      <td>Description of referencing secrets in stack configuration.</td>
+      <td><a href="./reference-secrets-in-stack-configuration.md">./reference-secrets-in-stack-configuration.md</a></td>
+    </tr>
+    <tr>
+      <td><mark style="color:purple;"><strong>Implement a custom stack component</strong></mark></td>
+      <td>Creating your custom stack component solutions.</td>
+      <td><a href="./implement-a-custom-stack-component.md">./implement-a-custom-stack-component.md</a></td>
+    </tr>
+    <tr>
+      <td><mark style="color:purple;"><strong>Implement a custom integration</strong></mark></td>
+      <td>Description of implementing a custom integration.</td>
+      <td><a href="./implement-a-custom-integration.md">./implement-a-custom-integration.md</a></td>
+    </tr>
+  </tbody>
+</table>
 
 <figure><img src="https://static.scarf.sh/a.png?x-pxid=f0b4f458-0a54-4fcd-aa95-d5ee424815bc" alt="ZenML Scarf"><figcaption></figcaption></figure>