test/e2e/upgrade/alert: Extend to testDuration, not just 1m #25912
Conversation
We're doing better in updates now, and want to ratchet down to bar critical-alert noise during updates. The old 1m alertPeriodCheckMinutes landed with this test in 3b8cb3c (Add CI test to check for crit alerts post upgrade, 2020-03-27, openshift#24786). DurationSinceStartInSeconds, which I'm using now, landed in ace1345 (test: Allow tests that check invariants over time to be constrained, 2021-01-06, openshift#25784).
We don't want any critical alerts firing. Watchdog is severity=none [1]. AlertmanagerReceiversNotConfigured is severity=warning [2]. KubeAPILatencyHigh was dropped in openshift/cluster-monitoring-operator#898, 2020-08-03, and was severity=warning anyway. [1]: https://github.com/openshift/cluster-monitoring-operator/blob/776379a9616be1cbea49dd86086d8be8230370ce/assets/prometheus-k8s/rules.yaml#L2326-L2336 [2]: https://github.com/openshift/cluster-monitoring-operator/blob/776379a9616be1cbea49dd86086d8be8230370ce/assets/prometheus-k8s/rules.yaml#L924-L933
KubeAPILatencyHigh was dropped in openshift/cluster-monitoring-operator#898, 2020-08-03, 4.6 [1]. [1]: https://bugzilla.redhat.com/show_bug.cgi?id=1846805#c13
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: wking The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/hold |
openshift-ci-robot
added
the
do-not-merge/hold
|
is intended to remove the ability to throw any alerts during upgrade |
|
@wking: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
@wking: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
3 participants
We're doing better in updates now, and want to ratchet down to bar critical-alert noise during updates. The old 1m
alertPeriodCheckMinuteslanded with this test in 3b8cb3c (#24786).DurationSinceStartInSeconds, which I'm using now, landed in ace1345 (#25784).I've also dropped some special-cased
alertnamefiltering, because we don't want any critical alerts firing. Watchdog isseverity=none. AlertmanagerReceiversNotConfigured isseverity=warning. KubeAPILatencyHigh was dropped in openshift/cluster-monitoring-operator#898, 4.6 and wasseverity=warninganyway.