Document Custom Compute Engine VMs

Fixes #794 Fixes #311 Fixes #101 Fixes #741 Fixes #793
cirruslabs · Feb 22, 2021 · 4ad565b · 4ad565b
1 parent e1e6baa
commit 4ad565b
Show file tree

Hide file tree

Showing 5 changed files with 75 additions and 13 deletions.
diff --git a/docs/guide/FreeBSD.md b/docs/guide/FreeBSD.md
@@ -14,7 +14,14 @@ task:
 
 !!! info "Under the Hood"
     Under the hood a simple integration with [Google Compute Engine](supported-computing-services.md#compute-engine) 
-    is used.
+    is used and basically `freebsd_instance` is a syntactic sugar for the following [`compute_engine_instance`](custom-vms.md) configuration:
+
+    ```yaml
+    compute_engine_instance:
+      image_project: freebsd-org-cloud-dev
+      image: family/freebsd-13-0
+      platform: freebsd
+    ```
 
 ## List of available image families
 

diff --git a/docs/guide/custom-vms.md b/docs/guide/custom-vms.md
@@ -0,0 +1,33 @@
+## Custom Compute Engine VMs
+
+Cirrus CI supports [many different compute services](supported-computing-services.md) when you bring your own infrastructure, 
+but internally at Cirrus Labs we use Google Cloud Platform for running all [managed by us instances](writing-tasks.md#execution-environment)
+except `macos_instance`. Already things like [Docker Builder](docker-builder-vm.md) and [`freebsd_instance`](FreeBSD.md)
+are basically a syntactic sugar for launching Compute Engine instances from a particular limited set of images.
+
+With `compute_engine_instance` it is possible to use any publicly available image for running your Cirrus tasks in.
+Such instances are particularly useful when you can't use Docker containers, for example, when you need to test things
+against newer versions of Linux kernel then the Docker host has.
+
+```yaml
+compute_engine_instance:
+  image_project: cirrus-images
+  image: family/docker-kvm # family or simply a full image name.
+  platform: linux
+  cpu: 4
+  memory: 16G
+  nested_virtualization: true # Whether to enable Intel VT-x. Defaults to false. 
+```
+
+# Building custom image for Compute Engine
+
+We recommend to use [Packer](https://www.packer.io/) for building your custom images. As an example, please take a look at [our Packer templates](https://github.com/cirruslabs/osx-images)
+used for building Docker Builder VM image.
+
+After building your image, please make sure [the image publicly available](https://cloud.google.com/compute/docs/images/managing-access-custom-images#share-images-publicly):
+
+```bash
+gcloud compute images add-iam-policy-binding $IMAGE_NAME \
+    --member='allAuthenticatedUsers' \
+    --role='roles/compute.imageUser'
+```
diff --git a/docs/guide/docker-builder-vm.md b/docs/guide/docker-builder-vm.md
@@ -47,6 +47,23 @@ For your convenience, a Docker Builder VM has some common packages pre-installed
 * Python
 * Ruby with Bundler
 
+### Under the hood
+
+Under the hood a simple integration with [Google Compute Engine](supported-computing-services.md#compute-engine)
+is used and basically `docker_builder` is a syntactic sugar for the following [`compute_engine_instance`](custom-vms.md) configuration:
+
+```yaml
+task:
+  compute_engine_instance:
+    image_project: cirrus-images
+    image: family/docker-builder
+    platform: linux	
+    cpu: 4
+    memory: 16G
+```
+
+You can check Packer templates of the VM image in [`cirruslabs/vm-images` repository](https://github.com/cirruslabs/vm-images).
+
 ### Layer Caching
 
 Docker has the `--cache-from` flag which allows using a previously built image as a cache source. This way only changed

diff --git a/docs/guide/writing-tasks.md b/docs/guide/writing-tasks.md
@@ -62,23 +62,27 @@ task:
 In order to specify where to execute a particular task you can choose from a variety of options by defining one of the
 following fields for a `task`:
 
-Field Name                 | Computing Service                                     | Description
--------------------------- | ----------------------------------------------------- | -----------------------
-`container`                | [Linux Community Cluster][container]                  | Linux Docker Container
-`windows_container`        | [Windows Community Cluster][windows_container]        | Windows Docker Container
-`macos_instance`           | [macOS Community Cluster][macos_instance]             | macOS Virtual Machines
-`freebsd_instance`         | [FreeBSD Community Cluster][freebsd_instance]         | FreeBSD Virtual Machines
-`gce_instance`             | [Google Compute Engine][gce_instance]                 | Linux, Windows and FreeBSD Virtual Machines in your GCP project
-`gke_container`            | [Google Kubernetes Engine][gke_container]             | Linux Docker Containers on private GKE cluster
-`ec2_instance`             | [Amazon Elastic Compute Cloud][ec2_instance]          | Linux Virtual Machines in your AWS
-`eks_instance`             | [Amazon Elastic Container Service][eks_instance]      | Linux Docker Containers on private EKS cluster
-`azure_container_instance` | [Azure Container Instances][azure_container_instance] | Linux and Windows Docker Container on Azure
-`anka_instance`            | [Anka Build by Veertu][anka_instance]                 | macOS VMs on your Anka Build
+Field Name                 | Managed by | Description
+-------------------------- | ---------- | -----------------------
+`container`                | **us**     | [Linux Docker Container][container]
+`windows_container`        | **us**     | [Windows Docker Container][windows_container]
+`docker_builder `          | **us**     | [Full-fledged VM pre-configured for running Docker][docker_builder]
+`macos_instance`           | **us**     | [macOS Virtual Machines][macos_instance]
+`freebsd_instance`         | **us**     | [FreeBSD Virtual Machines][freebsd_instance]
+`compute_engine_instance`  | **us**     | [Full-fledged custom VM][compute_engine_instance]
+`gce_instance`             | **you**    | [Linux, Windows and FreeBSD Virtual Machines in your GCP project][gce_instance]
+`gke_container`            | **you**    | [Linux Docker Containers on private GKE cluster][gke_container]
+`ec2_instance`             | **you**    | [Linux Virtual Machines in your AWS][ec2_instance]
+`eks_instance`             | **you**    | [Linux Docker Containers on private EKS cluster][eks_instance]
+`azure_container_instance` | **you**    | [Linux and Windows Docker Container on Azure][azure_container_instance]
+`anka_instance`            | **you**    | [macOS VMs on your Anka Build][anka_instance]
 
 [container]: linux.md
 [windows_container]: windows.md
+[docker_builder]: docker-builder-vm.md
 [macos_instance]: macOS.md
 [freebsd_instance]: FreeBSD.md
+[compute_engine_instance]: custom-vms.md
 [gce_instance]: supported-computing-services.md#compute-engine
 [gke_container]: supported-computing-services.md#kubernetes-engine
 [ec2_instance]: supported-computing-services.md#ec2

diff --git a/mkdocs.yml b/mkdocs.yml
@@ -95,6 +95,7 @@ nav:
         - 'Docker Builder on VM': guide/docker-builder-vm.md
         - 'Docker Builds on GKE': guide/docker-builds-on-kubernetes.md
         - 'Docker Pipe': guide/docker-pipe.md
+        - 'Custom VMs': guide/custom-vms.md
         - 'Computing Services': guide/supported-computing-services.md
       - 'Life of a Build': guide/build-life.md
       - 'Notifications': guide/notifications.md