Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kuadrant dependencies to OLM installation #6

Merged
merged 1 commit into from
Jan 26, 2022

Conversation

mikenairn
Copy link
Member

@mikenairn mikenairn commented Jan 24, 2022

Adds the generated kuadrant-controller installation manifests (CRD,RBAC,roles etc..) to the kuadrant-operator OLM bundle. This makes the controller installation part of the OLM installation of the operator itself.
Authorino and Limitador are added as OLM dependencies to the operator and defined in the bundles metadata/dependencies.yaml. The defined versions of these components must be available in a catalog source available to
the target namespace in order for the kuadrant-operator to install. For testing/dev purposes, the index image that is created from this repo make catalog-build, by default includes the latest supported version of authorino and limitador, and is what is used when running make deploy-olm.

Makefile updates:

  • Set Version to 0.0.0
  • Set IMAGE_TAG to latest for 0.0.0
  • Set REGISTRY(quay.io) and ORG(kuadrant) and use to build up IMAGE_TAG_BASE
  • Add authorino, limitador and controller dependencies kustomize config
    for development installations.
  • Add script to generate bundle dependencies file bundle/metadata/dependencies.yaml.
  • Add make target bundle-dependencies to update dependencies.yaml with correct versions form Makefile.
  • Update BUNDLE_IMGS to include limitadir and authorino bundles

Verification

Deploy kuadrant operator and dependency components using OLM catalog source on a local kind cluster:

$ make kind-create-cluster install-olm deploy-olm

Check the deployment:

$ kubectl get packagemanifest -l catalog=kuadrant-operator-catalog -n kuadrant-operator-system
NAME                 CATALOG              AGE
kuadrant-operator    Kuadrant Operators   81s
authorino-operator   Kuadrant Operators   81s
limitador-operator   Kuadrant Operators   81s
$ kubectl get crds | grep kuadrant.io
apiproducts.networking.kuadrant.io            2022-01-21T09:22:25Z
apis.networking.kuadrant.io                   2022-01-21T09:22:25Z
authconfigs.authorino.kuadrant.io             2022-01-21T09:22:19Z
authorinos.operator.authorino.kuadrant.io     2022-01-21T09:22:19Z
kuadrants.kuadrant.kuadrant.io                2022-01-21T09:22:25Z
limitadors.limitador.kuadrant.io              2022-01-21T09:22:21Z
ratelimits.limitador.kuadrant.io              2022-01-21T09:22:21Z
$ kubectl get subscription  -n kuadrant-operator-system
NAME                                                                          PACKAGE              SOURCE                      CHANNEL
authorino-operator-alpha-kuadrant-operator-catalog-kuadrant-operator-system   authorino-operator   kuadrant-operator-catalog   alpha
kuadrant                                                                      kuadrant-operator    kuadrant-operator-catalog   alpha
limitador-operator-alpha-kuadrant-operator-catalog-kuadrant-operator-system   limitador-operator   kuadrant-operator-catalog   alpha
$ kubectl get csv -n kuadrant-operator-system
NAME                        DISPLAY              VERSION   REPLACES   PHASE
authorino-operator.v0.0.0   Authorino Operator   0.0.0                Succeeded
kuadrant-operator.v0.0.0    Kuadrant             0.0.0                Succeeded
limitador-operator.v0.0.0   Limitador            0.0.0                Succeeded
$ kubectl get subscription kuadrant -n kuadrant-operator-system -o json | jq '.status.installedCSV'
"kuadrant-operator.v0.0.0"
$ kubectl get csv/kuadrant-operator.v0.0.0 -n kuadrant-operator-system -o json | jq '.status.message'
"install strategy completed with no errors"
$ kubectl get deployments -n kuadrant-operator-system
NAME                                    READY   UP-TO-DATE   AVAILABLE   AGE
authorino-operator-controller-manager   1/1     1            1           83s
kuadrant-controller-manager             1/1     1            1           79s
kuadrant-operator-controller-manager    1/1     1            1           79s
limitador-operator-controller-manager   1/1     1            1           76s
$ kubectl get pods -n kuadrant-operator-system
NAME                                                              READY   STATUS      RESTARTS   AGE
43539917290efc2e6f33b021e25da1f7a9f150d8d276acf92fd09ba5eb66srm   0/1     Completed   0          114s
6427bb5aac2edc4750e505a33be4eb4b56923d3aed4213abecdc7059d6tzk26   0/1     Completed   0          114s
aecaf21d9bde4883921cefdd0ba3a142f079e897473c5d40fd8a1ac9536czzn   0/1     Completed   0          114s
authorino-operator-controller-manager-b6896bc6b-9knhz             2/2     Running     0          96s
kuadrant-controller-manager-5ff75dd7c-kscc4                       2/2     Running     0          92s
kuadrant-operator-catalog-m4wv9                                   1/1     Running     0          2m19s
kuadrant-operator-controller-manager-8668d896f5-4b2jm             2/2     Running     0          92s
limitador-operator-controller-manager-cb94675db-qzvmd             2/2     Running     0          89s
$ kubectl get service -n kuadrant-operator-system
NAME                                                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
authorino-operator-controller-manager-metrics-service   ClusterIP   10.96.5.105     <none>        8443/TCP   2m57s
kuadrant-controller-manager-metrics-service             ClusterIP   10.96.172.117   <none>        8443/TCP   2m57s
kuadrant-operator-controller-manager-metrics-service    ClusterIP   10.96.97.187    <none>        8443/TCP   2m57s
limitador-operator-controller-manager-metrics-service   ClusterIP   10.96.239.36    <none>        8443/TCP   2m57s

@@ -3,7 +3,7 @@
# To re-generate a bundle for another specific version without changing the standard setup, you can:
# - use the VERSION as arg of the bundle target (e.g make bundle VERSION=0.0.2)
# - use environment variables to overwrite this value (e.g export VERSION=0.0.2)
VERSION ?= 0.0.1
VERSION ?= 0.0.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, i believe this should always be set to the latest released version of the operator, when we do a release, we bump this appropriately as part of that process, and commit it to main. Currently we do not have a release of this operator, so it's set to 0.0.0. Version here is the version of the OLM bundle, and must always be a semver, and can't be set to 'latest'.
Once we have an initial version, you can still install the latest version of the operator built from main, but you would need to override this value:

make deploy-olm VERSION=0.0.0 

We should also setup CI to build and publish this index image on 'main' changes so it is always kept in sync.

@guicassolato I know you made similar, but slightly different, changes for authorino, be good to get your opinion here and see if we can make this consistent across all operator repos.

LIMITADOR_OPERATOR_BUNDLE_IMG_TAG = v$(LIMITADOR_OPERATOR_BUNDLE_VERSION)
LIMITADOR_OPERATOR_GITREF = v$(LIMITADOR_OPERATOR_BUNDLE_VERSION)
endif
LIMITADOR_OPERATOR_BUNDLE_IMG ?= quay.io/kuadrant/limitador-operator-bundle:$(LIMITADOR_OPERATOR_BUNDLE_IMG_TAG)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, these should be pinned to a version once there are appropriate versions we can pin here. Installing the "latest" version of a sub component (authorino,limitador, controller) would still be possible by overriding the appropriate env var (AUTHORINO_OPERATOR_VERSION, LIMITADOR_OPERATOR_VERSION,KUADRANT_CONTROLLER_VERSION).

> config/dependencies/authorino/kustomization.yaml
envsubst \
< config/dependencies/limitador/kustomization.template.yaml \
> config/dependencies/limitador/kustomization.yaml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't figure out a better way of templating the kustomization file, but the point of this is to make this Makefile the source of truth for all kuadrant component versions, and allow you to override the version as required. Also not sure if envsubst works on macOS.

To install the latest authorino operator for example:

make dependencies-manifests AUTHORINO_OPERATOR_VERSION=latest
./bin/kustomize build config/dependencies/authorino | kubectl apply -f -

@@ -0,0 +1,2 @@
resources:
- github.com/Kuadrant/authorino-operator/config/default?ref=main
Copy link
Member Author

@mikenairn mikenairn Jan 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note, these dependency kustomizations are referencing the remote kustomization for that component rather than cloning and copying or expecting a generated set of manifests to exist in that repo.
Having the kustomizations work this way allows modifications to remote sets of manifests that aren't directly the responsibility of this repo. For example it would be possible for us to override the image being used for the authorino manager deployment in the same way we can modify this operators manager image:

$ cd config/dependencies/authorino && ../../../bin/kustomize edit set image quay.io/3scale/authorino-operator=quay.io/mnairn/authorino-operator:myfeature
$ cd ../../..
$ ./bin/kustomize build config/dependencies/authorino/ | grep image: 
              image:
        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.8.0
        image: quay.io/mnairn/authorino-operator:myfeature

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guicassolato If we were to take this approach with authorino and the authorino-operator would this get round potential issues you had with pinning the version of authorino on the authorino-operator main branch? i.e. If you want to install the latest version of authorino using the latest version of the operator, add a make target to authorino that overrides the authorino image and sets it to quay.io/kuadrant/authorino:latest

@eguzki
Copy link
Contributor

eguzki commented Jan 24, 2022

I am not sure about the fact that the kuadrant controller is installed by OLM. That prevents the kuadrant operator to customize the deployment of the kuadrant controller. By customize, I mean configure behavior settings, or credentials (TLS) if the controller ever needs for that. Or just the log level.

Authorino and Limitador are not deployed as part of the operator deployment. I think that the kuadrant controller shouldn't be either.

wdyt?

@mikenairn
Copy link
Member Author

mikenairn commented Jan 25, 2022

I am not sure about the fact that the kuadrant controller is installed by OLM. That prevents the kuadrant operator to customize the deployment of the kuadrant controller. By customize, I mean configure behavior settings, or credentials (TLS) if the controller ever needs for that. Or just the log level.

Authorino and Limitador are not deployed as part of the operator deployment. I think that the kuadrant controller shouldn't be either.

wdyt?

We could take the same approach as Limitador and Authorino and the CRDs/Roles here and create a deployment of the controller per kuadrant instance, and maybe that is preferable, and was something i looked at. I'd say the main reasons i decided not to do it that way, initially at least, is that it seemed odd to add CRDs to a cluster that nothing was going to reconcile(API and APIProduct), at least until someone created a kuadrant CR, and the ongoing talks into the future design of the kuadrant API suggesting a significant change to the controller in the near future.

I'm happy enough to drop the controller part of this PR out to a separate change request since it will require more work if we want to go that route now.

Adds the generated kuadrant-controller installation manifests (CRD,RBAC
roles etc..) to the kuadrant-operator OLM bundle. This makes the controller
installation part of the OLM installation of the operator itself.
Authorino and Limitador are added as OLM dependencies to the operator
and defined in the bundles metadata/dependencies.yaml. The defined versions of
these componenets must be available in a catalog source avaiolable to
the target namesapce in order for the kuadrant-operator to install.

Makefile updates:

* Set Version to 0.0.0
* Set IMAGE_TAG to latest for 0.0.0
* Set REGISTRY(quay.io) and ORG(kuadrant) and use to build up IMAGE_TAG_BASE
* Add authorino, limitador and controller dependencies kustomize config
  for development installations.
* Add script to generate bundle dependencies file bundle/metadata/dependencies.yaml.
* Add make target `bundle-dependencies` to update dependencies.yaml with correct versions form Makefile.
* Update BUNDLE_IMGS to include limitadir and authorino bundles
@mikenairn mikenairn merged commit bbd0e0f into Kuadrant:main Jan 26, 2022
@mikenairn mikenairn deleted the add_kuadrant_dependenices branch January 26, 2022 13:25
adam-cattermole pushed a commit to adam-cattermole/kuadrant-operator that referenced this pull request Jun 11, 2024
* istio being optional

* envoy patch policy controller

* envoy patch policy controller: fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants