Cache only required secrets by andfasano · Pull Request #903 · metal3-io/baremetal-operator

andfasano · 2021-05-31T17:31:42Z

This PR limits the BMO secrets caching just to those ones really required, by filtering secrets watch request through a new label selector.
This will fix the problem of the very high memory usage detected when BMO is configured to watch the entire cluster (WATCH_NAMESPACE=""), and moreover will keep in cache just the BMH related secrets.

Fixes #904

andfasano · 2021-05-31T17:32:01Z

/hold requires controller-runtime 0.9.0

andfasano · 2021-06-08T09:29:25Z

/hold cancel

controller-runtime 0.9.0 has been released

andfasano · 2021-06-08T09:45:31Z

/test-integration

furkatgofurov7 · 2021-06-08T18:36:10Z

/test-integration

andfasano · 2021-06-09T10:33:11Z

/hold investigating test-integration failure

andfasano · 2021-06-09T17:33:33Z

/hold cancel

andfasano · 2021-06-10T11:13:57Z

/test-integration

metal3-io-bot · 2021-06-10T17:24:20Z

@andfasano: you cannot LGTM your own PR.

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

andfasano · 2021-06-10T17:29:23Z

/test-integration

andfasano · 2021-06-11T16:03:58Z

/test-integration

andfasano · 2021-06-16T14:51:36Z

/assign @zaneb
cc @hardys

furkatgofurov7

There are conflicting changes with this and #917 in terms of go modules

andfasano · 2021-06-17T16:40:43Z

There are conflicting changes with this and #917 in terms of go modules

Which ones? I cross-checked them and it seems that the go.mod files are almost identical (all the changes here are due just the uplift of controller-runtime to 0.9.0). #917 also modifies the api go.mod

furkatgofurov7 · 2021-06-18T07:01:12Z

I cross-checked them and it seems that the go.mod files are almost identical

Yes that is what I meant actually, that both are doing the same changes in case if you were not aware of the other work.

zaneb

This looks basically good to me

andfasano · 2021-07-01T08:56:06Z

/test-integration

andfasano · 2021-07-07T07:40:37Z

/test-integration

zaneb · 2021-07-07T16:25:51Z

/lgtm
/test-integration

hardys · 2021-07-08T10:41:40Z

/approve

metal3-io-bot · 2021-07-08T10:41:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hardys

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [hardys]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Fixed manifests change (linebreaks) Fixed broken test due changes in handling finalizers in the fake client

The internal cache now filters out secrets without the label "metal3.io.environment":"baremetal". The controller takes care of applying such label to the owned secrets only

andfasano · 2021-07-08T12:08:38Z

Rebase was required due the CRD changes introduced in #884

andfasano · 2021-07-08T12:10:45Z

/test-integration

zaneb

/lgtm

zaneb · 2021-07-08T14:02:35Z

-                    description: Custom deploy method name. This name is specific to the deploy ramdisk used. If you don't have a custom deploy ramdisk, you shouldn't use CustomDeploy.
+                    description: Custom deploy method name. This name is specific
+                      to the deploy ramdisk used. If you don't have a custom deploy
+                      ramdisk, you shouldn't use CustomDeploy.


These changes should really be in the first commit.

metal3-io-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 31, 2021

metal3-io-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 31, 2021

andfasano force-pushed the dont-cache-secrets branch 2 times, most recently from 8525aca to ce32a2c Compare June 1, 2021 11:56

metal3-io-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 1, 2021

andfasano force-pushed the dont-cache-secrets branch from 90dd1a4 to 7cf62f1 Compare June 3, 2021 09:05

andfasano changed the title ~~[WIP] Cache only required secrets~~ Cache only required secrets Jun 3, 2021

metal3-io-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 3, 2021

zaneb reviewed Jun 7, 2021

View reviewed changes

Comment thread controllers/metal3.io/baremetalhost_controller.go Outdated

Comment thread controllers/metal3.io/baremetalhost_controller.go Outdated

Comment thread controllers/metal3.io/baremetalhost_controller_test.go Outdated

Comment thread go.mod Outdated

andfasano force-pushed the dont-cache-secrets branch from 7cf62f1 to ec6e1a1 Compare June 8, 2021 09:26

metal3-io-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 8, 2021

andfasano force-pushed the dont-cache-secrets branch from ec6e1a1 to fa83ffb Compare June 8, 2021 09:37

metal3-io-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 9, 2021

metal3-io-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 9, 2021

andfasano force-pushed the dont-cache-secrets branch from 2e407ef to a74ac8d Compare June 9, 2021 17:36

andfasano force-pushed the dont-cache-secrets branch 4 times, most recently from c31aaa8 to f145820 Compare June 11, 2021 12:03

metal3-io-bot assigned zaneb Jun 16, 2021

andfasano mentioned this pull request Jun 17, 2021

⚠️Prepare for CAPI v1alpha4 changes metal3-io/cluster-api-provider-metal3#167

Merged

furkatgofurov7 reviewed Jun 17, 2021

View reviewed changes

andfasano mentioned this pull request Jun 17, 2021

upgrade k8s/api to v0.21.1 (in apis/go.mod) and controller-tools to v0.6.0 #917

Merged

zaneb reviewed Jun 30, 2021

View reviewed changes

Comment thread controllers/metal3.io/baremetalhost_controller.go Outdated

Comment thread controllers/metal3.io/host_config_data.go Outdated

andfasano force-pushed the dont-cache-secrets branch from f145820 to 89aac5d Compare July 1, 2021 07:43

metal3-io-bot added the lgtm Indicates that a PR is ready to be merged. label Jul 7, 2021

metal3-io-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 8, 2021

andfasano added 2 commits July 8, 2021 07:04

Update controller-runtime 0.9.0

e86b039

Fixed manifests change (linebreaks) Fixed broken test due changes in handling finalizers in the fake client

Cache only baremetal hosts secrets

ec911ac

The internal cache now filters out secrets without the label "metal3.io.environment":"baremetal". The controller takes care of applying such label to the owned secrets only

andfasano force-pushed the dont-cache-secrets branch from 89aac5d to 9d02680 Compare July 8, 2021 12:01

metal3-io-bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 8, 2021

Apply label to userData/networkData/metaData secrets

931a00a

andfasano force-pushed the dont-cache-secrets branch from 9d02680 to 931a00a Compare July 8, 2021 12:05

zaneb reviewed Jul 8, 2021

View reviewed changes

metal3-io-bot added the lgtm Indicates that a PR is ready to be merged. label Jul 8, 2021

metal3-io-bot merged commit bef5ef5 into metal3-io:master Jul 8, 2021

Conversation

andfasano commented May 31, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andfasano commented May 31, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

andfasano commented Jun 8, 2021

Uh oh!

andfasano commented Jun 8, 2021

Uh oh!

furkatgofurov7 commented Jun 8, 2021

Uh oh!

andfasano commented Jun 9, 2021

Uh oh!

andfasano commented Jun 9, 2021

Uh oh!

andfasano commented Jun 10, 2021

Uh oh!

metal3-io-bot commented Jun 10, 2021

Uh oh!

andfasano commented Jun 10, 2021

Uh oh!

andfasano commented Jun 11, 2021

Uh oh!

andfasano commented Jun 16, 2021

Uh oh!

furkatgofurov7 left a comment

Choose a reason for hiding this comment

Uh oh!

andfasano commented Jun 17, 2021

Uh oh!

furkatgofurov7 commented Jun 18, 2021

Uh oh!

zaneb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

andfasano commented Jul 1, 2021

Uh oh!

andfasano commented Jul 7, 2021

Uh oh!

zaneb commented Jul 7, 2021

Uh oh!

hardys commented Jul 8, 2021

Uh oh!

metal3-io-bot commented Jul 8, 2021

Uh oh!

andfasano commented Jul 8, 2021

Uh oh!

andfasano commented Jul 8, 2021

Uh oh!

zaneb left a comment

Choose a reason for hiding this comment

Uh oh!

zaneb Jul 8, 2021

Choose a reason for hiding this comment

Uh oh!

andfasano Jul 8, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

andfasano commented May 31, 2021 •

edited

Loading