diff --git a/README.md b/README.md index 7e1ebd4..769baf1 100644 --- a/README.md +++ b/README.md @@ -50,6 +50,10 @@ The Flux Operator comes with a Kubernetes CRD called `FluxInstance`. A single cu can exist in a Kubernetes cluster with the name `flux` that must be created in the same namespace where the operator is deployed. +> [!NOTE] +> The `FluxInstance` API documentation is available at +> [docs/api/v1](https://github.com/controlplaneio-fluxcd/flux-operator/blob/main/docs/api/v1/fluxinstance.md). + ### Upstream Distribution To install the upstream distribution of Flux, create the following `FluxInstance` resource: diff --git a/docs/api/v1/fluxinstance.md b/docs/api/v1/fluxinstance.md new file mode 100644 index 0000000..ba8c77a --- /dev/null +++ b/docs/api/v1/fluxinstance.md @@ -0,0 +1,466 @@ +# FluxInstance + +**FluxInstance** is a declarative API for the installation, configuration +and automatic upgrade of the Flux distribution. + +A single custom resource of this kind can exist in a Kubernetes cluster +with the name `flux` that must be created in the same namespace +where the flux-operator is deployed. + +## Example + +The following example shows a FluxInstance custom resource that +installs the upstream Flux distribution with all available components, +and configures the flux-operator to automatically upgrade Flux +to the latest stable version: + +```yaml +apiVersion: fluxcd.controlplane.io/v1 +kind: FluxInstance +metadata: + name: flux + namespace: flux-system + annotations: + fluxcd.controlplane.io/reconcile: "enabled" + fluxcd.controlplane.io/reconcileEvery: "1h" + fluxcd.controlplane.io/reconcileTimeout: "3m" +spec: + distribution: + version: "2.x" + registry: "ghcr.io/fluxcd" + components: + - source-controller + - kustomize-controller + - helm-controller + - notification-controller + - image-reflector-controller + - image-automation-controller + cluster: + type: kubernetes + multitenant: false + networkPolicy: true + domain: "cluster.local" + storage: + class: "standard" + size: "10Gi" + kustomize: + patches: + - target: + kind: Deployment + name: "(kustomize-controller|helm-controller)" + patch: | + - op: add + path: /spec/template/spec/containers/0/args/- + value: --concurrent=10 + - op: add + path: /spec/template/spec/containers/0/args/- + value: --requeue-dependency=5s +``` + +You can run this example by saving the manifest into `fluxinstance.yaml`. + +1. Apply the resource on the cluster: + + ```shell + kubectl apply -f fluxinstance.yaml + ``` + +2. Run `kubectl get fluxinstance` to see the status of the resource: + + ```console + $ kubectl -n flux-system get fluxinstance + NAME AGE READY STATUS REVISION + flux 59s True Reconciliation finished in 52s v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 + ``` + +3. Run `kubectl describe fluxinstance` to see the reconciliation status components, conditions and events: + + ```console + $ kubectl -n flux-system describe fluxinstance flux + Status: + Components: + Digest: sha256:161da425b16b64dda4b3cec2ba0f8d7442973aba29bb446db3b340626181a0bc + Name: source-controller + Repository: ghcr.io/fluxcd/source-controller + Tag: v1.3.0 + Digest: sha256:48a032574dd45c39750ba0f1488e6f1ae36756a38f40976a6b7a588d83acefc1 + Name: kustomize-controller + Repository: ghcr.io/fluxcd/kustomize-controller + Tag: v1.3.0 + Digest: sha256:a67a037faa850220ff94d8090253732079589ad9ff10b6ddf294f3b7cd0f3424 + Name: helm-controller + Repository: ghcr.io/fluxcd/helm-controller + Tag: v1.0.1 + Digest: sha256:c0fab940c7e578ea519097d36c040238b0cc039ce366fdb753947428bbf0c3d6 + Name: notification-controller + Repository: ghcr.io/fluxcd/notification-controller + Tag: v1.3.0 + Digest: sha256:aed795c7a8b85bca93f6d199d5a14bbefaf925ad5aa5316b32a716cfa4070d0b + Name: image-reflector-controller + Repository: ghcr.io/fluxcd/image-reflector-controller + Tag: v0.32.0 + Digest: sha256:ab5097213194f3cd9f0e68d8a937d94c4fc7e821f6544453211e94815b282aa2 + Name: image-automation-controller + Repository: ghcr.io/fluxcd/image-automation-controller + Tag: v0.38.0 + Conditions: + Last Transition Time: 2024-06-03T12:20:57Z + Message: Reconciliation finished in 52s + Observed Generation: 1 + Reason: ReconciliationSucceeded + Status: True + Type: Ready + Last Applied Revision: v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 + Last Attempted Revision: v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal Progressing 6m20s flux-controller Installing revision v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 + Normal ReconciliationSucceeded 5m9s flux-controller Reconciliation finished in 52s + ``` + +4. Run `kubectl logs` on the flux-operator pod to see the reconciliation logs: + + ```shell + kubectl -n flux-system logs deployment/flux-operator + ``` + +5. Run `kubectl events` to see the events generated by the flux-operator: + + ```shell + kubectl -n flux-system events --for FluxInstance/flux + ``` + +6. Run `kubectl delete` to remove the FluxInstance resource and + to uninstall Flux without affecting any Flux-managed workloads: + + ```shell + kubectl -n flux-system delete FluxInstance/flux + ``` + +## Writing a FluxInstance spec + +As with all other Kubernetes config, a FluxInstance needs `apiVersion`, +`kind`, and `metadata` fields. The name of a FluxInstance object must be a +valid [DNS subdomain name](https://kubernetes.io/docs/concepts/overview/working-with-objects/names#dns-subdomain-names). + +A FluxInstance also needs a +[`.spec` section](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md#spec-and-status). + +### Distribution configuration + +The `.spec.distribution` field is required and specifies the Flux distribution to install. + +Example using the upstream Flux distribution: + +```yaml +spec: + distribution: + version: "2.x" + registry: "ghcr.io/fluxcd" +``` + +### Distribution version + +The `.spec.distribution.version` field is required and specifies the version of the Flux distribution to install. +The version field value must be a valid [semver](https://semver.org/) range or an exact version. + +Example using a semver range to configure the automatic upgrade +to the latest Flux minor version: + +```yaml +spec: + distribution: + version: "2.x" +``` + +Example using a semver range to configure the automatic upgrade +to the latest Flux patch version of the `2.3` series: + +```yaml +spec: + distribution: + version: "2.3.x" +``` + +Example using an exact version to install a specific Flux version: + +```yaml +spec: + distribution: + version: "2.3.0" +``` + +### Distribution registry + +The `.spec.distribution.registry` field is required and specifies the container registry +where the Flux distribution images are pulled from. + +Example using the upstream Flux distribution registry: + +```yaml +spec: + distribution: + version: "2.x" + registry: "ghcr.io/fluxcd" +``` + +### Distribution image pull secret + +The `.spec.distribution.imagePullSecret` field is optional and specifies the name of the Kubernetes secret +that contains the credentials to pull the Flux distribution images from a private registry. + +Example using the ControlPlane enterprise registry: + +```yaml +spec: + distribution: + version: "2.3.x" + registry: "ghcr.io/controlplaneio-fluxcd/distroless" + imagePullSecret: "flux-enterprise-auth" +``` + +The image pull secret must be created in the same namespace where the FluxInstance is deployed +and must be of type `kubernetes.io/dockerconfigjson`. + +Example generating a secret for the ControlPlane enterprise registry: + +```sh +kubectl create secret docker-registry flux-enterprise-auth \ + --namespace flux-system \ + --docker-server=ghcr.io \ + --docker-username=flux \ + --docker-password=$ENTERPRISE_TOKEN +``` + +### Components configuration + +The `.spec.components` field is optional and specifies the list of Flux components to install. + +When not specified, the operator will install the default set of components for the Flux distribution: + +```yaml +spec: + components: + - source-controller + - kustomize-controller + - helm-controller + - notification-controller +``` + +### Cluster configuration + +The `.spec.cluster` field is optional and specifies the Kubernetes cluster configuration. + +Example using the OpenShift cluster configuration: + +```yaml +spec: + cluster: + type: openshift + multitenant: true + networkPolicy: true + domain: "cluster.local" +``` + +### Cluster type + +The `.spec.cluster.type` field is optional and specifies the type of the Kubernetes cluster. +This field is used to enable specific configuration for AKS, EKS, GKE and OpenShift clusters. + +The supported values are `kubernetes` (default), `openshift`, `aks`, `eks` and `gke`. + +### Cluster multitenant + +The `.spec.cluster.multitenant` field is optional and specifies whether to enable Flux +[multi-tenancy lockdown](https://fluxcd.io/flux/installation/configuration/multitenancy/). + +### Cluster network policy + +The `.spec.cluster.networkPolicy` field is optional and specifies whether to restrict network access +to the Flux namespace from other namespaces. By default, network policy is enabled. + +### Cluster domain + +The `.spec.cluster.domain` field is optional and specifies the cluster internal domain name. +By default, the domain is set to `cluster.local`. + +### Storage configuration + +The `.spec.storage` field is optional and specifies the persistent storage for Flux internal artifacts. +When specified, the operator will create a persistent volume claim named `source-controller` with +the specified storage class and size and mount it to the Flux source-controller `/data` volume. + +### Storage class + +The `.spec.storage.class` field is required and specifies the storage class to use for the persistent volume claim. + +### Storage size + +The `.spec.storage.size` field is required and specifies the size of the persistent volume claim. + +### Kustomize patches + +The `.spec.kustomize.patches` field is optional and specifies the Kustomize patches to apply to the Flux controllers. + +Example: + +```yaml +spec: + kustomize: + patches: + - target: + kind: Deployment + name: "(kustomize-controller|helm-controller)" + patch: | + - op: add + path: /spec/template/spec/containers/0/args/- + value: --concurrent=10 + - op: add + path: /spec/template/spec/containers/0/args/- + value: --requeue-dependency=5s +``` + +### Reconciliation configuration + +The reconciliation behaviour can be configured using the following annotations: + +- `fluxcd.controlplane.io/reconcile`: Enable or disable the reconciliation loop. Default is `enabled`, set to `disabled` to pause the reconciliation. +- `fluxcd.controlplane.io/reconcileEvery`: Set the reconciliation interval. Default is `1h`. +- `fluxcd.controlplane.io/reconcileTimeout`: Set the reconciliation timeout. Default is `5m`. + +## FluxInstance Status + +### Conditions + +A FluxInstance enters various states during its lifecycle, reflected as Kubernetes Conditions. +It can be [reconciling](#reconciling-fluxinstance) while applying the +resources on the cluster, it can be [ready](#ready-fluxinstance), or it can [fail during +reconciliation](#failed-fluxinstance). + +The FluxInstance API is compatible with the **kstatus** specification, +and reports `Reconciling` and `Stalled` conditions where applicable to +provide better (timeout) support to solutions polling the Kustomization to +become `Ready`. + +#### Reconciling FluxInstance + +The flux-operator marks a FluxInstance as _reconciling_ when it starts +the reconciliation of the same. The Condition added to the FluxInstance's +`.status.conditions` has the following attributes: + +- `type: Reconciling` +- `status: "True"` +- `reason: Progressing` | `reason: ProgressingWithRetry` + +The Condition `message` is updated during the course of the reconciliation to +report the action being performed at any particular moment such as +building manifests, detecting drift, etc. + +The `Ready` Condition's `status` is also marked as `Unkown`. + +#### Ready FluxInstance + +The flux-operator marks a FluxInstance as _ready_ when the Flux configuration was +built and applied on the cluster and all health checks are observed to be passing. + +When the FluxInstance is "ready", the flux-operator sets a Condition with the +following attributes in the FluxInstance’s `.status.conditions`: + +- `type: Ready` +- `status: "True"` +- `reason: ReconciliationSucceeded` + +#### Failed FluxInstance + +The flux-operator may get stuck trying to reconcile and apply a +FluxInstance without completing. This can occur due to some of the following factors: + +- The specified distribution version is not available. +- The kustomization of the Flux components fails to build. +- Garbage collection fails. +- Running health checks fails. + +When this happens, the flux-operator sets the `Ready` Condition status to False +and adds a Condition with the following attributes to the FluxInstance’s +`.status.conditions`: + +- `type: Ready` +- `status: "False"` +- `reason: BuildFailed | HealthCheckFailed | ReconciliationFailed` + +The `message` field of the Condition will contain more information about why +the reconciliation failed. + +While the FluxInstance has one or more of these Conditions, the flux-operator +will continue to attempt a reconciliation with an +exponential backoff, until it succeeds and the FluxInstance is marked as [ready](#ready-fluxinstance). + +### Components status + +In order to provide visibility into the Flux components that are installed, +the flux-operator records the status of each component in the `.status.components` field, +including the image repository, tag and digest. + +Example: + +```text +Status: + Components: + Digest: sha256:161da425b16b64dda4b3cec2ba0f8d7442973aba29bb446db3b340626181a0bc + Name: source-controller + Repository: ghcr.io/fluxcd/source-controller + Tag: v1.3.0 + Digest: sha256:48a032574dd45c39750ba0f1488e6f1ae36756a38f40976a6b7a588d83acefc1 + Name: kustomize-controller + Repository: ghcr.io/fluxcd/kustomize-controller + Tag: v1.3.0 +``` + +### Inventory status + +In order to perform operations such as drift detection, garbage collection, upgrades, etc., +the flux-operator needs to keep track of all Kubernetes objects that are +reconciled as part of a FluxInstance. To do this, it maintains an inventory +containing the list of Kubernetes resource object references that have been +successfully applied and records it in `.status.inventory`. The inventory +records are in the format `Id: ___, V: `. + +Example: + +```text +Status: + Inventory: + Entries: + Id: flux-system_source-controller__ServiceAccount + V: v1 + Id: flux-system_source-controller__Service + V: v1 + Id: flux-system_source-controller_apps_Deployment + V: v1 +``` + +### Last applied revision + +`.status.lastAppliedRevision` is the last revision of the Flux distribution +that was successfully applied to the cluster. + +The revision is in the format `@sha256:`. + +The version is the Flux distribution exact semver version that was applied to the cluster. + +The digest is the SHA256 hash of the Flux distribution manifests and customisations +that was applied to the cluster. + +### Last attempted revision + +`.status.lastAttemptedRevision` is the last revision of the Flux distribution +that was attempted to be applied to the cluster. + +Example: + +```text +Status: + Last Applied Revision: v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 + Last Attempted Revision: v2.3.0@sha256:4cc5babdb1279ad0177bf513292deadbfa3f7b7c3da0be7fa53b39ab434f7219 +```