Skip to content

🌱 Enhance Tilt integration with CAPO using a ClusterClass template #1833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
k8s-ci-robot merged 1 commit into from
Jan 30, 2024
Merged

🌱 Enhance Tilt integration with CAPO using a ClusterClass template #1833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 96 additions & 1 deletion docs/book/src/development/development.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,104 @@ make docker-build docker-push

After generating `infrastructure-components.yaml`, replace the `us.gcr.io/k8s-artifacts-prod/capi-openstack/capi-openstack-controller:v0.3.4` with your image.

## Testing Cluster Creation using the 'dev-test' ClusterClass with Tilt

This guide demonstrates how to create a Kubernetes cluster using ClusterClass, specifically designed for a development environment. It includes initializing a Kind cluster, configuring secrets, and applying ClusterClass to deploy your cluster with Tilt.

### Creating a Kind Cluster

Create a Kind cluster. This process involves the initialization of Cluster API providers for OpenStack.

```bash
kind create cluster
export CLUSTER_TOPOLOGY=true
clusterctl init --infrastructure openstack
```

### Secret Configuration

CAPO needs a clouds.yaml file in order to manage the OpenStack resources needed for the Cluster. This should be supplied as a secret named `${CLUSTER_NAME}-cloud-config`. You can create this secret for example with:

```bash
kubectl create secret generic ${CLUSTER_NAME}-cloud-config --from-file=clouds.yaml
```

You can also make Tilt create it for you when you run `tilt up`, more on that in the next section.

## Developing with Tilt

We have support for using [Tilt](https://tilt.dev/) for rapid iterative development. Please visit the [Cluster API documentation on Tilt](https://cluster-api.sigs.k8s.io/developer/tilt.html) for information on how to set up your development environment.
We have support for using [Tilt](https://tilt.dev/) for rapid iterative development. Please visit the [Cluster API documentation on Tilt](https://cluster-api.sigs.k8s.io/developer/tilt.html) for information on how to set up your development environment.

Default values align with the devstack setup, as documented below. When specifying the `KUBERNETES_VERSION`, ensure an image named `ubuntu-2204-kube-{{ KUBERNETES_VERSION }}` is available in your environment, corresponding to that Kubernetes version.
For using Tilt with ClusterClass, update your `tilt-settings.yaml` file as described:

```yaml
kustomize_substitutions:
CLUSTER_TOPOLOGY: "true"
# Define the name of your cluster (e.g., "dev-test")
CLUSTER_NAME: "<cluster_name>"
# Desired Kubernetes Version for Your Cluster (e.g., "v1.28.5")
KUBERNETES_VERSION: "<kubernetes_version>"
# [Optional] SSH Keypair Name for Instances in OpenStack (Default: "")
OPENSTACK_SSH_KEY_NAME: "<openstack_keypair_name>"
# [Optional] Control Plane Machine Flavor (Default: m1.medium)
OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR: "<openstack_control_plane_machine_flavor>"
# [Optional] Node Machine Flavor (Default: m1.small)
OPENSTACK_NODE_MACHINE_FLAVOR: "<openstack_node_machine_flavor>"
# [Optional] OpenStack Cloud Environment (Default: capo-e2e)
OPENSTACK_CLOUD: "<openstack_cloud>"

additional_kustomizations:
secret_kustomization: /path/to/kustomize/secret/configuration
```

Ensure the specified path (`/path/to/kustomize/secret/configuration`) contains both the `clouds.yaml` file and a `kustomization.yaml` file. The `kustomization.yaml` should define the necessary resources, such as a Kubernetes secret, using the `clouds.yaml` file.

For example, if you want to automatically create a secret named `dev-test-cloud-config` with the content of your `clouds.yaml` every time you do `tilt up`, you could do the following.

Create a folder to hold the kustomization.
We will use `/tmp/capo-dev` as example here.

Add the `clouds.yaml` file that you want to use to the folder.
It could look something like this:

```yaml
clouds:
capo-e2e:
auth:
username: demo
password: secretadmin
# If using application credentials you would have something like this instead:
# auth_type: v3applicationcredential
# application_credential_id: abc123
# application_credential_secret: 456def
user_domain_id: default
auth_url: https://example.com/identity
domain_id: default
project_name: demo
verify: false
region_name: RegionOne
```

Create a kustomization file named `kustomization.yaml` in the same folder:

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Do not add random hash to the end of the secret name
generatorOptions:
disableNameSuffixHash: true
secretGenerator:
- files:
- clouds.yaml
name: dev-test-cloud-config
type: Opaque
```

If you now add `/tmp/capo-dev` to the `additional_kustomizations`, tilt will automatically apply
the secret when you do `tilt up`.

To check that the kustomization produces the desired output, do `kustomize build /tmp/capo-dev`.

## Running E2E tests locally

Expand Down
13 changes: 13 additions & 0 deletions templates/cluster-template-development.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
name: ${CLUSTER_NAME}
spec:
topology:
class: dev-test
version: ${KUBERNETES_VERSION}
workers:
machineDeployments:
- class: default-worker
name: md-0
replicas: 1
128 changes: 128 additions & 0 deletions templates/clusterclass-dev-test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
apiVersion: cluster.x-k8s.io/v1beta1
kind: ClusterClass
metadata:
name: dev-test
spec:
controlPlane:
ref:
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlaneTemplate
name: ${CLUSTER_NAME}-control-plane-template
machineInfrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackMachineTemplate
name: ${CLUSTER_NAME}-control-plane-machine-template
infrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackClusterTemplate
name: ${CLUSTER_NAME}-openstackcluster-template
workers:
machineDeployments:
- class: default-worker
template:
bootstrap:
ref:
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
name: ${CLUSTER_NAME}-default-worker-bootstraptemplate
infrastructure:
ref:
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackMachineTemplate
name: ${CLUSTER_NAME}-default-worker-machine-template
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
name: ${CLUSTER_NAME}-default-worker-bootstraptemplate
spec:
template:
spec:
files: []
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
provider-id: openstack:///'{{ instance_id }}'
name: '{{ local_hostname }}'
---
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlaneTemplate
metadata:
name: ${CLUSTER_NAME}-control-plane-template
spec:
template:
spec:
kubeadmConfigSpec:
clusterConfiguration:
apiServer:
extraArgs:
cloud-provider: external
controllerManager:
extraArgs:
cloud-provider: external
files: []
initConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
provider-id: openstack:///'{{ instance_id }}'
name: '{{ local_hostname }}'
joinConfiguration:
nodeRegistration:
kubeletExtraArgs:
cloud-provider: external
provider-id: openstack:///'{{ instance_id }}'
name: '{{ local_hostname }}'
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackClusterTemplate
metadata:
name: ${CLUSTER_NAME}-openstackcluster-template
spec:
template:
spec:
apiServerLoadBalancer:
enabled: true
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
dnsNameservers:
- 8.8.8.8
identityRef:
kind: Secret
name: ${CLUSTER_NAME}-cloud-config
managedSecurityGroups: true
nodeCidr: 10.6.0.0/24
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackMachineTemplate
metadata:
name: ${CLUSTER_NAME}-control-plane-machine-template
spec:
template:
spec:
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
flavor: ${OPENSTACK_CONTROL_PLANE_MACHINE_FLAVOR:=m1.medium}
identityRef:
kind: Secret
name: ${CLUSTER_NAME}-cloud-config
image:
name: ubuntu-2204-kube-${KUBERNETES_VERSION}
sshKeyName: ${OPENSTACK_SSH_KEY_NAME:=""}
---
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha8
kind: OpenStackMachineTemplate
metadata:
name: ${CLUSTER_NAME}-default-worker-machine-template
spec:
template:
spec:
cloudName: ${OPENSTACK_CLOUD:=capo-e2e}
flavor: ${OPENSTACK_NODE_MACHINE_FLAVOR:=m1.small}
identityRef:
kind: Secret
name: ${CLUSTER_NAME}-cloud-config
image:
name: ubuntu-2204-kube-${KUBERNETES_VERSION}
sshKeyName: ${OPENSTACK_SSH_KEY_NAME:=""}