Skip to content

Commit

Permalink
docs: document operator update path... (#216)
Browse files Browse the repository at this point in the history
... and compatibility matrix

---------

Signed-off-by: Mario Constanti <[email protected]>
  • Loading branch information
bavarianbidi authored Nov 20, 2024
1 parent 114eda2 commit 5351af8
Show file tree
Hide file tree
Showing 6 changed files with 236 additions and 15 deletions.
21 changes: 11 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,17 @@

### Garm Version

garm-operator uses [`garm-api-client`](https://github.com/cloudbase/garm/tree/main/client) to talk
with `garm` servers. The supported `garm` server version is determined by `garm-api-client`.
`garm-operator` is tightly coupled with the `garm` server. Therefore, the compatibility between `garm-operator` and `garm` is crucial.

> [!WARNING]
> `garm-operator` is currently tested with `garm` server version `v0.1.4`. Newer versions are not guaranteed to work.
>
> `garm` server version `>v0.1.4` introduces a few breaking changes.
> [provider]() and [credential]() configuration has moved away from being defined in the `garm` server configuration.
> Instead, they are now defined via the `garm` API itself.
> This feature set isn't implemented in the `garm-operator` yet.
| garm-operator | garm | compatible |
|---------------|---------|--------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `0.4.x` | `0.1.5` | :white_check_mark: |
| `0.4.x` | `0.1.4` | :x: |
| `0.3.x` | `0.1.4` | :white_check_mark: |
| `0.3.x` | `0.1.5` | :warning: It is not recommended to operate during a longer time period. This state mostly exists when you update the `garm-operator` from `0.3.x` to `0.4.x` |

> [!NOTE]
> please also read the [operator update guide](docs/operator_update.md) when you want to update the `garm-operator`.
### Kubernetes Version

Expand Down Expand Up @@ -92,7 +93,7 @@ For local development, please read the [development guide](DEVELOPMENT.md).

To make some assumptions and corresponding decisions transparent, we use ADRs (Architecture Decision Records) to document them.

All ADRs can be found in the [here](docs/architectural-decision-records.md).
All ADRs can be found [here](docs/architectural-decision-records.md).

## Contributing

Expand Down
48 changes: 48 additions & 0 deletions docs/adrs/github_default_endpoint.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
<!-- SPDX-License-Identifier: MIT -->

---
date: 2024-11-20
desc: Reflection of the default GitHub endpoint
state: accepted
---
<!--
What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.?
-->

<!--
This is a basic ADR template from [Documenting architecture decisions - Michael Nygard](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions).
It's possible to manage the ADR files with [adr-tools](https://github.com/npryce/adr-tools).
-->
# Reflection of the default GitHub endpoint

## Context and Problem Statement

With `garm` in version `0.1.5`, the GitHub API endpoints aren't defined in the configuration file anymore. The GitHub API endpoints are now defined via the `garm` API.

`garm` also ships with a default immutable GitHub API endpoint and points to the public GitHub API.

With the new API version `v1beta1` for all `garm-operator` related resources, we have to make use of the new `GitHubEndpoint` object to define the connection to the GitHub API.
The `GitHubEndpoint` is referenced in the `GitHubCredential` object.

## Decision Drivers

* The default GitHub API endpoint is immutable.
* The GitHub API endpoint must be configured via a new `GitHubEndpoint` object which then got referenced in the `GitHubCredential` object.

## Considered Options

* Reflecting the default GitHub API endpoint back into the kubernetes cluster by creating a `GitHubEndpoint` object with the default values.
But as the default GitHub API endpoint is immutable, we have to catch this case in the operator and forbid any update/delete operations on this reflected `GitHubEndpoint`.

## Pros and Cons of the Options

#### Pros:
* The code must not handle the immutability of the default GitHub API endpoint.

#### Cons:
* Everyone who wants to use the `garm-operator` with the default GitHub API endpoint has to create a `GitHubEndpoint` object.

## Decision Outcome

The validation rules in the `CRD` blocks the creation of a `GitHubEndpoint` object with the name `github.com`.
8 changes: 5 additions & 3 deletions docs/architectural-decision-records.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@

In this file we are collecting all our DHC CaaS architectural decision records.

| Date | Description |
|:-----------|:----------------------------------------------------|
| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) |
| Date | Description |
|:-----------|:-----------------------------------------------------------------------------|
| 2023-07-24 | [reduce login API calls](adrs/jwt_token_caching.md) |
| 2023-12-04 | [Garm Server Initialization](adrs/garm_server_initialization.md) |
| 2024-11-20 | [Reflection of the default GitHub endpoint](adrs/github_default_endpoint.md) |
4 changes: 2 additions & 2 deletions docs/kube-state-metrics/kube-state-metrics-config.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<!-- SPDX-License-Identifier: MIT -->

# Kube State Metrics Configuration
# kube-state-metrics Configuration

[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for a kube-state-metrics agent to expose metrics of `garm-operators` custom resources.
[Here](../../config/kube-state-metrics/configmap.yaml) you will find a sample configuration for [kube-state-metrics](https://github.com/kubernetes/kube-state-metrics) to expose metrics of `garm-operators` custom resources.
If you are using the official [helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-state-metrics) you can place the contents of `.data.config.yaml` into your helm-charts `values.yaml` file like so:

```yaml
Expand Down
161 changes: 161 additions & 0 deletions docs/operator_update.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
<!-- SPDX-License-Identifier: MIT -->

# compatibility

<!-- toc -->
- [update to operator 0.4.x](#update-to-operator-04x)
- [1. update <code>garm</code> to version <code>&gt;=0.1.5</code>](#1-update-garm-to-version-015)
- [2. update <code>garm-operator</code> to version <code>0.4.x</code>](#2-update-garm-operator-to-version-04x)
- [3. create new <code>CustomResources</code>](#3-create-new-customresources)
<!-- /toc -->

## update to operator 0.4.x

> [!WARNING]
> `garm-operator` in version `0.4.x` is not compatible with `garm` in version `0.1.4`.
> If you connect `garm-operator` in version `0.4.x` to `garm` in version `0.1.4`,
> the operator will stop working as we check the `garm` version in the operator.
### 1. update `garm` to version `>=0.1.5`

`garm-operator` in version `0.3.x` is compatible with `garm` in version `0.1.5`, if the `garm` server got updated from `0.1.4` to `0.1.5`.
This is because `garm` is doing some migration steps by moving parts of the configuration from the `garm` server to the `garm` API.
These newly introduced API endpoints are not by `garm-operator` in version `0.3.x`.

Therefore it's possible to still work with the existing `CustomResources`.

### 2. update `garm-operator` to version `0.4.x`

Once `garm` got updated to version `0.1.5`, you can update the `garm-operator` to version `0.4.x`.

### 3. create new `CustomResources`

As `garm` moved some configuration parts to the API, you have to create a few new `CustomResources`.
Without these new `CustomResources`, `garm-operator` is not able to reconcile `Enterprises`, `Organizations` and `Repository` objects.

#### 3.1 create a `GarmServerConfig` object

It's now possible to define the `callbackUrl`, `metadataUrl` and `webhookUrl` via a `GarmServerConfig` object. Therefore it's not needed anymore to restart the `garm` server to apply these changes.

```yaml
apiVersion: garm-operator.mercedes-benz.com/v1beta1
kind: GarmServerConfig
metadata:
name: garm-server-config
namespace: garm-operator-system
spec:
callbackUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/callbacks
metadataUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/metadata
webhookUrl: http://<fqdn-to-your-garm-server-instance>/api/v1/webhook
```
By running `kubectl get garmserverconfig` you should see the newly created `GarmServerConfig` object.

```bash
NAME ID VERSION AGE
garm-server-config dd2524b9-0789-499d-ba7d-3dba65cf9d3f v0.1.5 16h
```

More information about the defined Urls can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#controller-operations).

#### 3.2 create `GitHubEndpoint` objects

The `GitHubEndpoint` objects are used to define the connection to the GitHub API. It's possible to define multiple `GitHubEndpoint` objects to connect to different GitHub instances.

```yaml
apiVersion: garm-operator.mercedes-benz.com/v1beta1
kind: GitHubEndpoint
metadata:
name: github-enterprise
namespace: garm-operator-system
spec:
description: "github"
apiBaseUrl: "https://github.com"
uploadBaseUrl: "https://uploads.github.com"
baseUrl: "https://api.github.com/"
```

By running `kubectl get githubendpoint` you should see the newly created `GitHubEndpoint` object.

```bash
NAME URL READY AGE
github https://github.com True 3m2s
```

More information about the github endpoints can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-endpoints)

> [!NOTE]
> `garm` already ships a default `Github` object. But as this object is immutable, and we do not wanted to reflect the object into the `garm-operator`, we decided to create a new `CustomResource` for this. See the [Reflection of the default GitHub endpoint ADR](docs/adrs/github_default_endpoint.md) for more information.

#### 3.2 create `GitHubCredential` objects

With the new `v1beta1` of `Enterprises`, `Organizations` or `Repositories`, the reference to the Github credential has changed.
In the previous `v1alpha1`, the used credential was referenced by the `.spec.credentialsName` field, which then pointed to a credential object, specified in the `garm` server configuration.

With the new `v1beta1`, the reference to the credential is done by the `.spec.credentialRef` field, which points to a `GitHubCredential` object.

> [!NOTE]
> As long as `GitHubCredential` is not applied in `v1beta1`, the `garm-operator` will quit reconciling `Enterprises`, `Organizations` or `Repositories`.
> The current assumption is, that a `GitHubCredential`, which was set in `.spec.credentialsName` in `v1alpha1`, will be created as `GitHubCredential` in `v1beta1` with the same name.
> If this is not the case, the `garm-operator` will not be able to reconcile the `Enterprises`, `Organizations` or `Repositories` objects.
> This state can be seen in the `status.conditions` field of these objects.
>
> ```yaml
> status:
> conditions:
> - lastTransitionTime: "2024-11-14T20:01:27Z"
> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found
> reason: CredentialsRefFailed
> status: "False"
> type: Ready
> - lastTransitionTime: "2024-11-14T20:01:27Z"
> message: GitHubCredential.garm-operator.mercedes-benz.com "github-pat" not found
> reason: CredentialsRefFailed
> status: "False"
> type: CredentialsRef
> [...]
> ```

Create the `GitHubCredential` object and a corresponding `Secret` object with the PAT token.

```yaml
apiVersion: garm-operator.mercedes-benz.com/v1beta1
kind: GitHubCredential
metadata:
name: github-pat
namespace: garm-operator-system
spec:
description: PAT for github
endpointRef:
apiGroup: garm-operator.mercedes-benz.com
kind: GitHubEndpoint
name: github
authType: pat
secretRef:
name: github-pat
key: token
---
apiVersion: v1
kind: Secret
metadata:
name: github-pat
namespace: garm-operator-system
data:
token: <base64 encoded PAT>
```

More information about the github credentials can be found [in the garm documentation](https://github.com/cloudbase/garm/blob/main/doc/using_garm.md#github-credentials)

By running `kubectl get githubcredential` you should see the newly created `GitHubCredential` object.

```bash
NAME ID READY AUTHTYPE GITHUBENDPOINT AGE
github-pat 1 True pat github 3m51s
```

By running `kubectl get githubcredential -o wide` you see all the `Enterprise`, `Organization` or `Repository` objects which are using this `GitHubCredential`.

```bash
NAME ID READY ERROR AUTHTYPE GITHUBENDPOINT REPOSITORIES ORGANIZATIONS ENTERPRISES AGE
github-pat 1 True pat github ["my-org"] 4m46s
```
9 changes: 9 additions & 0 deletions docs/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,20 @@

<!-- toc -->
- [Quickstart](#quickstart)
- [<a href="architectural-decision-records.md">Architecture Decision Records</a>](#architecture-decision-records)
- [how to](#how-to)
- [scale runners](#scale-runners)
- [pause reconciliation](#pause-reconciliation)
- [<a href="config/configuration-parsing.md">configure the operator</a>](#configure-the-operator)
- [<a href="kube-state-metrics/kube-state-metrics-config.md">monitor operator CRs</a>](#monitor-operator-crs)
<!-- /toc -->

## Quickstart

[Quickstart](quickstart.md) contains a walk-through of how to get started with `garm-operator`.

## [Architecture Decision Records](architectural-decision-records.md)

## how to

### scale runners
Expand Down Expand Up @@ -97,3 +102,7 @@ $ kubectl annotate organization developer-org garm-operator.mercedes-benz.com/pa

organization.garm-operator.mercedes-benz.com/developer-org annotate
```

### [configure the operator](config/configuration-parsing.md)

### [monitor operator CRs](kube-state-metrics/kube-state-metrics-config.md)

0 comments on commit 5351af8

Please sign in to comment.