Add script, templates and manifests to make RHOBS deployable on test clusters #112

matej-g · 2022-01-11T15:40:07Z

This PR intends to allow deployment of the RHOBS stack outside of our typical depolyment scenario, which is using this repository in conjunction with the AppSRE Interface.

In order to allow for easier deployment for testing purposes, this PR contains additional resources that allow for the RHOBS to be deployed on any OpenShift cluster by replacing external dependencies (objects provided by AppSRE Interface - secrets, service accounts, object storage, OIDC provider). The additional resources are included in the tests directory, which includes:

Secrets (e.g. object storage), service accounts and role bindings which replace objects normally provided by AppSRE or are obtained from Vault
New templates to generate manifests to replace external dependencies with local alternatives (Minio to replace S3, Dex to replace Red Hat's SSO)
Files with template parameter overrides (also usually handled by AppSRE Interface)

On top of the additional manifests, there is a convenience script provided to easily deploy (and tear down) all manifests and templates.

Not included

Presently, the change set does include only 'core' parts of the RHOBS, meaning some templates / namespaces / objects are not part of the test deployment:

Logs related namespaces are not included, as the aim is to first allow testing for metrics
Jaeger-related deployments are also excluded, as they functionally are not part of the RHOBS
MST namespace is not included, since virtually it is a copy of the Observatorium namespace(s) with modified configuration
Services are not exposed, meaning OpenShift routes are not part of the deployment - these need to be added by the user, depending on particular test case - either oc expose can be use or a directly port forwarding with oc port-forward is possible as well

Signed-off-by: Matej Gera <[email protected]>

moadz

LGTM! Thanks for this.

resources/services/observatorium-metrics-template.yaml

moadz · 2022-01-11T16:51:19Z

resources/services/telemeter-template.yaml

+    authorize_url: aHR0cHM6Ly9hcGkuc3RhZ2Uub3BlbnNoaWZ0LmNvbS9hcGkvYWNjb3VudHNfbWdtdC92MS9jbHVzdGVyX3JlZ2lzdHJhdGlvbnM=
+    client_id: dGVzdA==
+    client_secret: WlhoaGJYQnNaUzFoY0hBdGMyVmpjbVYw
+    oidc_issuer: aHR0cDovL2RleC5kZXguc3ZjLmNsdXN0ZXIubG9jYWw6NTU1Ni9kZXg=


Are these placeholders? If not we probably shouldn't check these in.

They are the base64 encodings of...
authorize_url is https://api.stage.openshift.com/api/accounts_mgmt/v1/cluster_registrations
oidc_issuer is http://dex.dex.svc.cluster.local:5556/dex

So I guess the actual values

It's either test credentials which should not be really sensitive or publicly known / accessible resources like the OCM staging endpoint to register cluster.

moadz · 2022-01-11T16:54:03Z

services/observatorium.libsonnet

+      stringData+: {
+        'client-id': 'test',
+        'client-secret': 'ZXhhbXBsZS1hcHAtc2VjcmV0',
+        'issuer-url': 'http://dex.dex.svc.cluster.local:5556/dex',


'issuer-url': 'http://dex.dex.svc.cluster.local:5556/dex',

dupe or just weird local address?

lol what's weird about this address?

double dex :)

I think one is namespace and other is service name, right?

tests/README.md

moadz · 2022-01-12T10:26:47Z

tests/README.md

+The parameter files can be edited accordingly to accomodate your specific testing scenario.
+
+### How to
+To deploy the RHOBS stack on a cluster, use the `launch.sh` script from within this directory. Run:


nit: is it worth including links to docs/instructions on how best to requisition a cluster. (I know we already have this documented elsewhere)

Also, is the plan to have a long-lived cluster for rhobs testing, or to create one transiently?

+1 to that!

I would guess that this is for transient clusters as a long-lived test cluster would be deployed via app-interface just like stage and prod are.

I'll add links to more resources, good idea! 👍

moadz · 2022-01-12T10:27:29Z

tests/README.md

@@ -0,0 +1,33 @@
+## Deploying RHOBS for testing


This limits us to a single test 'instance' per project/cluster right?

At the moment yes but we can add support for more instance I guess in the next iteration if we'd like to

Signed-off-by: Matej Gera <[email protected]>

bill3tt

This is great stuff @matej-g - I'm looking forward to being able to run a testing cluster to poke around with 🤗

bill3tt · 2022-01-12T11:55:58Z

resources/services/telemeter-template.yaml

+    authorize_url: aHR0cHM6Ly9hcGkuc3RhZ2Uub3BlbnNoaWZ0LmNvbS9hcGkvYWNjb3VudHNfbWdtdC92MS9jbHVzdGVyX3JlZ2lzdHJhdGlvbnM=
+    client_id: dGVzdA==
+    client_secret: WlhoaGJYQnNaUzFoY0hBdGMyVmpjbVYw
+    oidc_issuer: aHR0cDovL2RleC5kZXguc3ZjLmNsdXN0ZXIubG9jYWw6NTU1Ni9kZXg=


They are the base64 encodings of...
authorize_url is https://api.stage.openshift.com/api/accounts_mgmt/v1/cluster_registrations
oidc_issuer is http://dex.dex.svc.cluster.local:5556/dex

So I guess the actual values

bill3tt · 2022-01-12T11:56:55Z

services/observatorium.libsonnet

+      stringData+: {
+        'client-id': 'test',
+        'client-secret': 'ZXhhbXBsZS1hcHAtc2VjcmV0',
+        'issuer-url': 'http://dex.dex.svc.cluster.local:5556/dex',


lol what's weird about this address?

bill3tt · 2022-01-12T11:58:50Z

tests/README.md

+The parameter files can be edited accordingly to accomodate your specific testing scenario.
+
+### How to
+To deploy the RHOBS stack on a cluster, use the `launch.sh` script from within this directory. Run:


+1 to that!

I would guess that this is for transient clusters as a long-lived test cluster would be deployed via app-interface just like stage and prod are.

bill3tt · 2022-01-12T12:02:09Z

My only comment would be to hold off merging until members of the team have had the chance to go through the whole flow so we can find and catch any bugs at source.

bill3tt · 2022-01-25T10:36:23Z

@matej-g I would say that this is ready to merge 💪

matej-g · 2022-01-25T10:39:05Z

@matej-g I would say that this is ready to merge muscle

I'm on this today, just need to go through comments to see what's up and I'm merging 👍

Co-authored-by: Moad Zardab <[email protected]>

matej-g · 2022-02-02T13:42:33Z

This has been marinating long enough, it's good to be merged and we can potentially iterate on this further in future PRs.

matej-g added 8 commits January 11, 2022 11:47

Adjust tenant and session secrets

0126386

Signed-off-by: Matej Gera <[email protected]>

Update image names and tags

f135729

Signed-off-by: Matej Gera <[email protected]>

Add Minio and Dex jsonnet templates

09410aa

Signed-off-by: Matej Gera <[email protected]>

Additional test secrets

d656bef

Signed-off-by: Matej Gera <[email protected]>

Add test service accounts and role bindings

587bb7a

Signed-off-by: Matej Gera <[email protected]>

Add generated templates

9c6102b

Signed-off-by: Matej Gera <[email protected]>

Add parameter override files

ceb2046

Signed-off-by: Matej Gera <[email protected]>

Add launch script and README

fca360f

Signed-off-by: Matej Gera <[email protected]>

moadz approved these changes Jan 12, 2022

View reviewed changes

matej-g force-pushed the rhobs-deploy-latest branch 2 times, most recently from bc1882e to 4569ccb Compare January 12, 2022 10:44

Cleanup

00aa258

Signed-off-by: Matej Gera <[email protected]>

matej-g force-pushed the rhobs-deploy-latest branch from 4569ccb to 00aa258 Compare January 12, 2022 10:58

bill3tt reviewed Jan 12, 2022

View reviewed changes

Update tests/README.md

055aa42

Co-authored-by: Moad Zardab <[email protected]>

matej-g marked this pull request as ready for review January 26, 2022 14:21

Merge remote-tracking branch 'origin/main' into rhobs-deploy-latest

629e42a

matej-g merged commit 9601c75 into rhobs:main Feb 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add script, templates and manifests to make RHOBS deployable on test clusters #112

Add script, templates and manifests to make RHOBS deployable on test clusters #112

matej-g commented Jan 11, 2022

moadz left a comment

moadz Jan 11, 2022

bill3tt Jan 12, 2022

matej-g Jan 25, 2022

moadz Jan 11, 2022

bill3tt Jan 12, 2022

moadz Jan 24, 2022

matej-g Jan 25, 2022

moadz Jan 12, 2022

bill3tt Jan 12, 2022

matej-g Jan 25, 2022

moadz Jan 12, 2022

matej-g Jan 25, 2022

bill3tt left a comment

bill3tt Jan 12, 2022

bill3tt Jan 12, 2022

bill3tt Jan 12, 2022

bill3tt commented Jan 12, 2022

bill3tt commented Jan 25, 2022

matej-g commented Jan 25, 2022

matej-g commented Feb 2, 2022

Add script, templates and manifests to make RHOBS deployable on test clusters #112

Add script, templates and manifests to make RHOBS deployable on test clusters #112

Conversation

matej-g commented Jan 11, 2022

Not included

moadz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bill3tt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bill3tt commented Jan 12, 2022

bill3tt commented Jan 25, 2022

matej-g commented Jan 25, 2022

matej-g commented Feb 2, 2022