Bug 2078528: bump cluster-node-tuning-operator apirequests upper bound for OpenStack by EmilienM · Pull Request #27052 · openshift/origin

EmilienM · 2022-04-25T15:10:11Z

When deploying OCP on OpenStack with OVNKubernetes, we got this error
from the tests:

"Operator \"cluster-node-tuning-operator\" produces more watch requests than expected: watchrequestcount=143, upperbound=88, ratio=1.625"

Bumping the upperbound to 103 should solve this issue.

openshift-ci · 2022-04-25T15:10:29Z

@EmilienM: This pull request references Bugzilla bug 2078528, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 2078528: bump cluster-node-tuning-operator apirequests upper bound for OpenStack

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

EmilienM · 2022-04-25T15:12:05Z

/bugzilla refresh

openshift-ci · 2022-04-25T15:12:15Z

@EmilienM: This pull request references Bugzilla bug 2078528, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.0) matches configured target release for branch (4.11.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mdbooth

Do we know the source of all these additional watch events? Have we reached out to the owners of cluster-node-tuning-operator? It's my understanding that, in aggregate, the cost of these things is non-trivial, so we could just be sweeping a potential issue under the carpet.

If we're going to bump the limit I'd like to see an associated justification for it beyond "it's breaking CI".

MarSik · 2022-04-25T16:14:06Z

Well PAO was merged with NTO, but the limits were supposed to be bumped already.

yanirq · 2022-04-26T07:22:58Z

Well PAO was merged with NTO, but the limits were supposed to be bumped already.

The limits where indeed bumped but not for OpenStack: #26944

dulek · 2022-04-26T09:05:30Z

/test e2e-openstack

dulek · 2022-04-26T09:05:46Z

/test e2e-openstack-ovn

openshift-ci · 2022-04-26T09:05:52Z

@dulek: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test e2e-aws-fips
/test e2e-aws-image-registry
/test e2e-aws-jenkins
/test e2e-aws-serial
/test e2e-gcp
/test e2e-gcp-builds
/test e2e-gcp-image-ecosystem
/test e2e-gcp-upgrade
/test extended_gssapi
/test extended_ldap_groups
/test extended_networking
/test images
/test verify
/test verify-deps

The following commands are available to trigger optional jobs:

/test e2e-agnostic-cmd
/test e2e-aws
/test e2e-aws-cgroupsv2
/test e2e-aws-csi
/test e2e-aws-csi-migration
/test e2e-aws-disruptive
/test e2e-aws-multitenant
/test e2e-aws-ovn
/test e2e-aws-proxy
/test e2e-aws-single-node
/test e2e-aws-single-node-serial
/test e2e-aws-single-node-upgrade
/test e2e-aws-upgrade
/test e2e-azure
/test e2e-gcp-csi
/test e2e-gcp-disruptive
/test e2e-gcp-fips-serial
/test e2e-gcp-ovn-rt-upgrade
/test e2e-metal-ipi
/test e2e-metal-ipi-ovn-dualstack
/test e2e-metal-ipi-ovn-ipv6
/test e2e-metal-ipi-serial
/test e2e-metal-ipi-virtualmedia
/test e2e-openstack
/test e2e-openstack-serial
/test e2e-vsphere
/test okd-e2e-gcp

Use /test all to run the following jobs that were automatically triggered:

pull-ci-openshift-origin-master-e2e-agnostic-cmd
pull-ci-openshift-origin-master-e2e-aws-cgroupsv2
pull-ci-openshift-origin-master-e2e-aws-csi
pull-ci-openshift-origin-master-e2e-aws-fips
pull-ci-openshift-origin-master-e2e-aws-serial
pull-ci-openshift-origin-master-e2e-aws-single-node
pull-ci-openshift-origin-master-e2e-aws-single-node-upgrade
pull-ci-openshift-origin-master-e2e-gcp
pull-ci-openshift-origin-master-e2e-gcp-csi
pull-ci-openshift-origin-master-e2e-gcp-ovn-rt-upgrade
pull-ci-openshift-origin-master-e2e-gcp-upgrade
pull-ci-openshift-origin-master-images
pull-ci-openshift-origin-master-verify
pull-ci-openshift-origin-master-verify-deps

Details

In response to this:

/test e2e-openstack-ovn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

EmilienM · 2022-04-26T12:01:35Z

/test e2e-openstack

mandre · 2022-04-26T13:06:15Z

test/extended/apiserver/api_requests.go

 				"cluster-image-registry-operator":        112,
 				"cluster-monitoring-operator":            48,
-				"cluster-node-tuning-operator":           44.0,
+				"cluster-node-tuning-operator":           143.0,


@stbenjam, since you bumped the allowed watches for node-tuning-operator for other platforms in #26944, does this number makes sense to you?
I'm unable to run your script to check what are the openstack percentiles.

Finally managed to generate the percentiles. This is what we get for OpenStack:

50% 90% 95% 100% 123 147 154 187

Looks like we need to take the highest value, add 10%, and divide by 2. For us that would be (187 + 19) / 2 = 103.

…d for OpenStack When deploying OCP on OpenStack with OVNKubernetes, we got this error from the tests: ``` "Operator \"cluster-node-tuning-operator\" produces more watch requests than expected: watchrequestcount=143, upperbound=88, ratio=1.625" ``` Bumping the upperbound to 103 should solve this issue.

openshift-ci · 2022-04-27T13:12:17Z

@EmilienM: This pull request references Bugzilla bug 2078528, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.0) matches configured target release for branch (4.11.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

Details

In response to this:

Bug 2078528: bump cluster-node-tuning-operator apirequests upper bound for OpenStack

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mandre

/lgtm

mdbooth · 2022-04-27T13:15:41Z

My previous concern has been addressed.

/lgtm

mandre · 2022-04-27T14:35:55Z

/test e2e-openstack

stbenjam · 2022-04-27T16:30:27Z

Looks accurate based on the history, and similar to values for other platforms.

/approve

openshift-ci · 2022-04-27T16:30:53Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: EmilienM, mandre, mdbooth, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~test/extended/OWNERS~~ [stbenjam]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2022-04-27T16:33:42Z

@EmilienM: All pull requests linked via external trackers have merged:

openshift/origin#27052

Bugzilla bug 2078528 has been moved to the MODIFIED state.

Details

In response to this:

Bug 2078528: bump cluster-node-tuning-operator apirequests upper bound for OpenStack

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2022-04-27T16:34:05Z

@EmilienM: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-gcp-ovn-rt-upgrade	`7875d10`	link	false	`/test e2e-gcp-ovn-rt-upgrade`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

EmilienM · 2022-04-27T16:40:05Z

/cherry-pick release-4.10

openshift-cherrypick-robot · 2022-04-27T16:43:49Z

@EmilienM: new pull request created: #27064

Details

In response to this:

/cherry-pick release-4.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci bot added bugzilla/severity-unspecified Referenced Bugzilla bug's severity is unspecified for the PR. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Apr 25, 2022

openshift-ci bot requested review from adambkaplan and knobunc April 25, 2022 15:10

mdbooth suggested changes Apr 25, 2022

View reviewed changes

mandre reviewed Apr 26, 2022

View reviewed changes

mandre reviewed Apr 27, 2022

View reviewed changes

openshift-ci bot assigned mdbooth Apr 27, 2022

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 27, 2022

openshift-ci bot assigned mandre Apr 27, 2022

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 27, 2022

openshift-merge-robot merged commit ac3c998 into openshift:master Apr 27, 2022

EmilienM deleted the bz2078528 branch April 27, 2022 16:33

openshift-cherrypick-robot mentioned this pull request Apr 27, 2022

[release-4.10] Bug 2079520: bump cluster-node-tuning-operator apirequests upper bound for OpenStack #27064

Closed

Conversation

EmilienM commented Apr 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Apr 25, 2022

Uh oh!

EmilienM commented Apr 25, 2022

Uh oh!

openshift-ci bot commented Apr 25, 2022

Uh oh!

mdbooth left a comment

Choose a reason for hiding this comment

Uh oh!

MarSik commented Apr 25, 2022

Uh oh!

yanirq commented Apr 26, 2022

Uh oh!

dulek commented Apr 26, 2022

Uh oh!

dulek commented Apr 26, 2022

Uh oh!

openshift-ci bot commented Apr 26, 2022

Uh oh!

EmilienM commented Apr 26, 2022

Uh oh!

mandre Apr 26, 2022

Choose a reason for hiding this comment

Uh oh!

mandre Apr 27, 2022

Choose a reason for hiding this comment

Uh oh!

mandre Apr 27, 2022

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Apr 27, 2022

Uh oh!

mandre left a comment

Choose a reason for hiding this comment

Uh oh!

mdbooth commented Apr 27, 2022

Uh oh!

mandre commented Apr 27, 2022

Uh oh!

stbenjam commented Apr 27, 2022

Uh oh!

openshift-ci bot commented Apr 27, 2022

Uh oh!

openshift-ci bot commented Apr 27, 2022

Uh oh!

openshift-ci bot commented Apr 27, 2022

Uh oh!

EmilienM commented Apr 27, 2022

Uh oh!

openshift-cherrypick-robot commented Apr 27, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

EmilienM commented Apr 25, 2022 •

edited

Loading