test/extended/prometheus: Validate alerting rules #26476
Conversation
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
/assign @simonpasquier |
|
/retest |
bison
changed the title
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
|
/retest |
1 similar comment
|
/retest |
dgrisonnet
left a comment
dgrisonnet
left a comment
There was a problem hiding this comment.
Awesome, I just have a few comments, but it looks good to me overall.
| } | ||
|
|
||
| if alertingRules == nil { | ||
| var err error |
There was a problem hiding this comment.
nit: we could have fetchAlertingRules initializing the error variable since it isn't used anywhere else.
There was a problem hiding this comment.
I was trying to avoid fetching these each time, while still being able to separate each check into its own g.It("should..") block. So alertingRules is actually defined in the outer scope, and it only does the fetch if it hasn't been initialized. Having the call to fetchAlertingRules() initialize err would shadow alertingRules. Let me know if I'm misunderstanding though.
There was a problem hiding this comment.
Ah indeed, I missed that alertingRules was defined in the outer scope, makes sense now 👍
The [Alerting Consistency][1] enhancement, and the proposed updates to it in [openshift/enhancements #897][2], define a style-guide for the alerts shipped as part of OpenShift. This adds a test validating some of the guidelines considered required. [1]: https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/alerting-consistency.md [2]: openshift/enhancements#897
|
/retest |
|
/lgtm |
|
I guess we need someone from the OWNERS file to approve this. Could maybe @bparees or @smarterclayton have a look if you have a minute? |
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bison, bparees, dgrisonnet The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
bot
added
the
approved
|
/retest-required |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
5 similar comments
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
18 similar comments
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
@bison: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
3 similar comments
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
/retest-required Please review the full test history for this PR and help us cut down flakes. |
|
TRT had to revert this today because it was causing a large number of jobs to fail. In the unrevert, please fix or add exceptions for the jobs that were failing. A list of job runs that contained this failure is available here: https://sippy.ci.openshift.org/sippy-ng/jobs/4.10/runs?filters=%7B%22items%22%3A%5B%7B%22columnField%22%3A%22failedTestNames%22%2C%22operatorValue%22%3A%22contains%22%2C%22value%22%3A%22OpenShift%20alerting%20rules%20should%20have%20description%20and%20summary%20annotations%22%7D%5D%2C%22linkOperator%22%3A%22and%22%7D&sort=desc&sortField=timestamp It appears to be all OVN and vsphere jobs. |
The [OpenShift Alerting Consistency][1] enhancement defines a style guide for the alerts shipped as part of OpenShift. This adds a test validating some of the guidelines considered required. This was originally added in openshift#26476, but was reverted in openshift#26499 due to failures with OVN and vSphere clusters. This adds the tests back, but adds exceptions for the non-compliant alerts as well as marking the failing tests as flakes for now. We'll gather data and make the tests required once we're reasonably sure things are passing with all the existing alerts. [1]: https://github.com/openshift/enhancements/blob/master/enhancements/monitoring/alerting-consistency.md
8 participants
The Alerting Consistency enhancement, and the proposed updates to
it in openshift/enhancements #897, define a style-guide for the
alerts shipped as part of OpenShift. This adds a test validating some
of the guidelines considered required.
This builds on and replaces #26016. There are currently a lot of failures due to non-conformant rules. We'll be filing bugs for these, and will add exceptions here until they are fixed.