Extend CVO alerts to cover update retrieval #357
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openshift-ci-robot
added
the
do-not-merge/work-in-progress
openshift-ci-robot
added
the
approved
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
a80d7eb to
ca9a94a
Compare
Contributor
Author
|
/test e2e-aws |
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
5010537 to
2d99330
Compare
Contributor
Author
|
/test e2e-aws |
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
1487e3b to
a9f575a
Compare
wking
reviewed
Apr 23, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
Apr 23, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
Apr 23, 2020
wking
reviewed
Apr 23, 2020
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
a9f575a to
31b703e
Compare
jottofar
force-pushed
the
ota-151-extend-alerts
branch
2 times, most recently
from
1c9b7dd to
6aaa85a
Compare
jottofar
changed the title
openshift-ci-robot
removed
the
do-not-merge/work-in-progress
Contributor
Author
|
/retest |
1 similar comment
Contributor
Author
|
/retest |
Contributor
Author
|
/test e2e-aws |
wking
added a commit
to wking/openshift-release
that referenced
this pull request
Apr 30, 2020
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786
Contributor
Author
|
/test e2e-aws |
wking
reviewed
May 4, 2020
install/0000_90_cluster-version-operator_02_servicemonitor.yaml
Outdated
Show resolved
Hide resolved
wking
reviewed
May 4, 2020
jottofar
force-pushed
the
ota-151-extend-alerts
branch
8 times, most recently
from
b81e6f0 to
3cc123d
Compare
Contributor
Author
|
/test images |
Contributor
Author
|
/retest |
Added metric cluster_version_operator_update_retrieval_timestamp_seconds to track last known successful update retrieval time and use metric cluster_operator_conditions to determine update retrieval failure reason. Added alert CannotRetrieveUpdates that fires when metric cluster_version_operator_update_retrieval_timestamp_seconds is >= 3600 unless reason, as reported by cluster_operator_conditions is NoChannel. Alert reports last known successful update retrieval time, reason unable to retrieve updates, and console URL to get more information.
jottofar
force-pushed
the
ota-151-extend-alerts
branch
from
3cc123d to
1005398
Compare
Contributor
Author
|
/test e2e-aws-upgrade |
1 similar comment
Contributor
Author
|
/test e2e-aws-upgrade |
Member
|
🎉 /lgtm |
Contributor
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jottofar, wking The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Contributor
Author
|
/test e2e-aws-upgrade |
Contributor
|
/retest Please review the full test history for this PR and help us cut down flakes. |
2 similar comments
Contributor
|
/retest Please review the full test history for this PR and help us cut down flakes. |
Contributor
|
/retest Please review the full test history for this PR and help us cut down flakes. |
Contributor
Author
|
/test e2e-aws-upgrade |
Contributor
|
/retest Please review the full test history for this PR and help us cut down flakes. |
1 similar comment
Contributor
|
/retest Please review the full test history for this PR and help us cut down flakes. |
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631
stbenjam
added a commit
to stbenjam/dev-scripts
that referenced
this pull request
Jun 3, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631 Co-authored-by: W. Trevor King <[email protected]>
openshift-merge-robot
pushed a commit
to openshift-metal3/dev-scripts
that referenced
this pull request
Sep 23, 2021
CI and nightly releases are not part of the official Red Hat Cincinnati graphs. This commit removes the channel property [1], which will result in the NoChannel condition [2], but will keep the CVO from attempting to find its current version in the official Cincinnati [3]. That in turn should keep the CVO from throwing a new, critical alert [4], which will keep it from running afoul of recent update e2e logic that forbids critical alerts [5]. See this related PR [6] which does this for other platforms IPI install step. [1]: https://github.com/openshift/installer/blob/4eca2efd615f8abd65f576721e2410b19f0d40d0/data/data/manifests/bootkube/cvo-overrides.yaml.template#L8 [2]: https://github.com/openshift/cluster-version-operator/blob/fa452c2d270f1f989f3868ef97ae8cf825713583/docs/user/status.md#nochannel [3]: https://bugzilla.redhat.com/show_bug.cgi?id=1827378#c4 [4]: openshift/cluster-version-operator#357 [5]: openshift/origin#24786 [6]: openshift/release#8631 Co-authored-by: W. Trevor King <[email protected]> Co-authored-by: W. Trevor King <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.