Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes #815

openshift-cherrypick-robot · 2022-08-04T22:46:43Z

This is an automated cherry-pick of #806

/assign wking

In some clusters, these PromQL queries can hang for hours, possibly forever [1]. I think we have a 30s default KeepAlive timeout [2], but apparently there's enough socket traffic to keep from tripping that. This adds a 5m cap to the PromQL calls, although I'm not particularly attached to that particular number. We can always raise it if we start seeing timeouts in Insights for queries where taking that long seems reasonable. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=2109374#c12 [2]: https://pkg.go.dev/github.com/prometheus/client_golang/api#pkg-variables

openshift-ci · 2022-08-04T22:46:58Z

@openshift-cherrypick-robot: Bugzilla bug 2109374 has been cloned as Bugzilla bug 2115564. Retitling PR to link against new bug.
/retitle [release-4.11] Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes

Details

In response to this:

[release-4.11] Bug 2109374: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci · 2022-08-04T22:47:14Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 2115564, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.0) matches configured target release for branch (4.11.0)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 2109374 is in the state VERIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
dependent Bugzilla bug 2109374 targets the "4.12.0" release, which is one of the valid target releases: 4.12.0
bug has dependents

Requesting review from QA contact:
/cc @shellyyang1989

Details

In response to this:

[release-4.11] Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking · 2022-08-04T22:50:03Z

GitHub tracks the target-branch as structured information for the pull requests, and folks reading 4.11 release notes will know that the changes are 4.11 specific, so drop the redundant [release-4.11] from the pull-request title:

/retitle Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes

wking · 2022-08-04T22:51:26Z

I moved the bug to 4.11.z, so the Bugzilla bot will be grumpy now until 4.11 GAs.

/bugzilla refresh

openshift-ci · 2022-08-04T22:51:32Z

@wking: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

I moved the bug to 4.11.z, so the Bugzilla bot will be grumpy now until 4.11 GAs.

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking

/lgtm

openshift-bot · 2022-08-05T01:01:04Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-05T01:01:09Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

LalatenduMohanty

/lgtm

LalatenduMohanty · 2022-08-05T20:07:08Z

/test unit

LalatenduMohanty · 2022-08-05T20:07:41Z

/label backport-risk-assessed

openshift-ci · 2022-08-05T20:10:05Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LalatenduMohanty, openshift-cherrypick-robot, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [LalatenduMohanty,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci · 2022-08-05T20:19:23Z

@openshift-cherrypick-robot: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

openshift-bot · 2022-08-06T01:01:06Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-06T01:01:11Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2022-08-07T01:01:03Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-07T01:01:08Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2022-08-08T01:01:06Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-08T01:01:13Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

shellyyang1989 · 2022-08-08T07:15:53Z

/label qe-approved

openshift-bot · 2022-08-09T01:01:10Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-09T01:01:16Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-bot · 2022-08-10T01:01:06Z

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

openshift-ci · 2022-08-10T01:01:18Z

@openshift-bot: This pull request references Bugzilla bug 2115564, which is invalid:

expected the bug to target the "4.11.0" release, but it targets "4.11.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking · 2022-08-10T20:47:16Z

openshift/release#31215 has landed.

/bugzilla refresh

openshift-ci · 2022-08-10T20:47:24Z

@wking: This pull request references Bugzilla bug 2115564, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.11.z) matches configured target release for branch (4.11.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 2109374 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
dependent Bugzilla bug 2109374 targets the "4.12.0" release, which is one of the valid target releases: 4.12.0
bug has dependents

Requesting review from QA contact:
/cc @shellyyang1989

Details

In response to this:

openshift/release#31215 has landed.

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

jiajliu · 2022-08-16T00:46:07Z

/label cherry-pick-approved

openshift-ci · 2022-08-16T00:50:37Z

@openshift-cherrypick-robot: All pull requests linked via external trackers have merged:

openshift/cluster-version-operator#815

Bugzilla bug 2115564 has been moved to the MODIFIED state.

Details

In response to this:

Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking · 2022-08-18T03:24:22Z

Verified.

/cherrypick release-4.10

openshift-cherrypick-robot · 2022-08-18T03:25:02Z

@wking: new pull request created: #825

Details

In response to this:

Verified.

/cherrypick release-4.10

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci bot assigned wking Aug 4, 2022

openshift-cherrypick-robot mentioned this pull request Aug 4, 2022

Bug 2109374: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes #806

Merged

openshift-ci bot changed the title ~~[release-4.11] Bug 2109374: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes~~ [release-4.11] Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes Aug 4, 2022

openshift-ci bot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Aug 4, 2022

openshift-ci bot requested review from jottofar, shellyyang1989 and vrutkovs August 4, 2022 22:47

openshift-ci bot changed the title ~~[release-4.11] Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes~~ Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes Aug 4, 2022

openshift-ci bot added bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. and removed bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Aug 4, 2022

wking approved these changes Aug 4, 2022

View reviewed changes

openshift-ci bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 4, 2022

LalatenduMohanty approved these changes Aug 5, 2022

View reviewed changes

openshift-ci bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Aug 5, 2022

openshift-ci bot assigned LalatenduMohanty Aug 5, 2022

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Aug 8, 2022

openshift-ci bot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Aug 10, 2022

openshift-ci bot added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Aug 16, 2022

openshift-merge-robot merged commit 56a6ae0 into openshift:release-4.11 Aug 16, 2022

openshift-cherrypick-robot mentioned this pull request Aug 18, 2022

OCPBUGS-235: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes #825

Merged

Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes #815

Bug 2115564: pkg/clusterconditions/promql: Cap PromQL queries at 5 minutes #815

Uh oh!

Conversation

openshift-cherrypick-robot commented Aug 4, 2022

Uh oh!

openshift-ci bot commented Aug 4, 2022

Uh oh!

openshift-ci bot commented Aug 4, 2022

Uh oh!

wking commented Aug 4, 2022

Uh oh!

wking commented Aug 4, 2022

Uh oh!

openshift-ci bot commented Aug 4, 2022

Uh oh!

wking left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-bot commented Aug 5, 2022

Uh oh!

openshift-ci bot commented Aug 5, 2022

Uh oh!

LalatenduMohanty left a comment

Choose a reason for hiding this comment

Uh oh!

LalatenduMohanty commented Aug 5, 2022

Uh oh!

LalatenduMohanty commented Aug 5, 2022

Uh oh!

openshift-ci bot commented Aug 5, 2022

Uh oh!

openshift-ci bot commented Aug 5, 2022

Uh oh!

openshift-bot commented Aug 6, 2022

Uh oh!

openshift-ci bot commented Aug 6, 2022

Uh oh!

openshift-bot commented Aug 7, 2022

Uh oh!

openshift-ci bot commented Aug 7, 2022

Uh oh!

openshift-bot commented Aug 8, 2022

Uh oh!

openshift-ci bot commented Aug 8, 2022

Uh oh!

shellyyang1989 commented Aug 8, 2022

Uh oh!

openshift-bot commented Aug 9, 2022

Uh oh!

openshift-ci bot commented Aug 9, 2022

Uh oh!

openshift-bot commented Aug 10, 2022

Uh oh!

openshift-ci bot commented Aug 10, 2022

Uh oh!

wking commented Aug 10, 2022

Uh oh!

openshift-ci bot commented Aug 10, 2022

Uh oh!

jiajliu commented Aug 16, 2022

Uh oh!

openshift-ci bot commented Aug 16, 2022

Uh oh!

wking commented Aug 18, 2022

Uh oh!

openshift-cherrypick-robot commented Aug 18, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants