OSDOCS-2428: Adding CVO conditions

[sagidlow]

Applies to 4.7+
QE ack required.
SME ack required.
Preview Link: Understanding CVO conditions
JIRA Link: https://issues.redhat.com/browse/OSDOCS-2428

[netlify]

✔️ Deploy Preview for osdocs ready!

🔨 Explore the source changes: 5c377fe

🔍 Inspect the deploy log: https://app.netlify.com/sites/osdocs/deploys/61a63a50ed7a0e0008439369

😎 Browse the preview: https://deploy-preview-35453--osdocs.netlify.app/openshift-enterprise/latest/updating/understanding-the-update-service

[sagidlow]

Requesting review from @wking, @sdodson, @jottofar

[wking]

I left a bunch of nits inline, but overall looks good to me, and will be nice to have these in our official docs :). Apologies for the nits and stale bits that you're inheriting from the CVO's local docs... Among which, I don't think this current list is complete, but haven't run a full audit of our current code. We could do that audit, but it would take some time, and I dunno if we need to be complete before landing something is useful. I'm also not clear if this is really an openshift-docs thing, or if we'd be better off landing this in runbooks, where we can link it from the alerts like CannotRetrieveUpdates that will go off when cluster is sad. Recent CVO work has overhauled those alerts, and perhaps things like this are now sufficiently self-describing that we don't need to cover them independently in docs? Or maybe there are some subset of especially-confusing types/reasons that are worth calling out in docs, beyond what can be covered by runbooks and alerts?

[sagidlow]

I left a bunch of nits inline, but overall looks good to me, and will be nice to have these in our official docs :). Apologies for the nits and stale bits that you're inheriting from the CVO's local docs... Among which, I don't think this current list is complete, but haven't run a full audit of our current code. We could do that audit, but it would take some time, and I dunno if we need to be complete before landing something is useful. I'm also not clear if this is really an openshift-docs thing, or if we'd be better off landing this in runbooks, where we can link it from the alerts like CannotRetrieveUpdates that will go off when cluster is sad. Recent CVO work has overhauled those alerts, and perhaps things like this are now sufficiently self-describing that we don't need to cover them independently in docs? Or maybe there are some subset of especially-confusing types/reasons that are worth calling out in docs, beyond what can be covered by runbooks and alerts?

@wking, the content drastically changed since the last time you looked at it. I pulled out all the stale information, and mostly did it based on these two links:

• https://github.com/openshift/enhancements/blob/master/dev-guide/cluster-version-operator/dev/clusteroperator.md#conditions
• https://pkg.go.dev/github.com/openshift/api/config/v1#ClusterStatusConditionType

Would love to get your new thoughts, if you have time :)

[sdodson]

LGTM, lets get this in.

[sagidlow]

@sdodson Who is the QE that should review this?

[sdodson]

@jianlinliu is QE for OTA team, I'm unsure if they cover CVO / ClusterOperator definitions as well.

[sagidlow]

@jianlinliu Could you review this from the QE side? TY

[mburke5678]

s/cluster-version operator/Cluster Version Operator (CVO)

[mburke5678]

Where do these conditions appear?

[sagidlow]

@sdodson - If I'm not mistaken the CVO conditions appear during an upgrade in the Log right? I'm not 100% sure but thought you might know :)

[sdodson]

Sorry for delay in getting to this, they are reflected in the clusterversion object, which can be used programmatically, parts of which are expressed in the Cluster Settings page during upgrades.
oc get clusterversion -o yaml

[mburke5678]

a Degraded state represents persistent observation of a condition
Is condition the right word here? It seems odd that the Degraded condition reports on a condition. Maybe problem or error or such??

[sdodson]

Maybe just drop "of a condition" all together?

[mburke5678]

I'm not sure what missing condition means. The CV is not reporting any Upgradeable condition?

[sdodson]

It's just saying that in absence of Upgradeable=False it's assumed to be Upgradeable. Some operators may set Unknown or not set this condition while initializing, though hopefully that's brief.

[mburke5678]

@sagidlow A few more nits. Otherwise LGTM

[jianlinliu]

The ClusterOperator is a custom resource object which holds the current state of the Operator. The ClusterOperator conveys the state to the rest of the cluster.

The topic is talking about CVO conditions, but the main description is talking about "CO".

And go through the whole PR, sounds like there is no much description about the relationship and dependency between the status of COs and the conditions of CVO, then how it affect the final upgrade operations.

And per my knowledge, CVO collects and summarizes the status of all COs, then determinate CVO conditions, sometimes, even CVO also set some conditions by itself. For example:

            "status": {
                "availableUpdates": null,
                "conditions": [
                    {
                        "lastTransitionTime": "2021-10-21T00:46:18Z",
                        "message": "Done applying 4.8.0-0.nightly-2021-10-20-155651",
                        "status": "True",
                        "type": "Available"
                    },
                    {
                        "lastTransitionTime": "2021-10-21T00:46:00Z",
                        "status": "False",
                        "type": "Failing"
                    },
                    {
                        "lastTransitionTime": "2021-10-21T00:46:18Z",
                        "message": "Cluster version is 4.8.0-0.nightly-2021-10-20-155651",
                        "status": "False",
                        "type": "Progressing"
                    },
                    {
                        "lastTransitionTime": "2021-10-21T00:11:57Z",
                        "message": "Unable to retrieve available updates: currently reconciling cluster version 4.8.0-0.nightly-2021-10-20-155651 not found in the \"stable-4.8\" channel",
                        "reason": "VersionNotFound",
                        "status": "False",
                        "type": "RetrievedUpdates"
                    },
                    {
                        "lastTransitionTime": "2021-10-21T00:12:28Z",
                        "message": "Kubernetes 1.22 and therefore OpenShift 4.9 remove several APIs which require admin consideration. Please see\nthe knowledge article https://access.redhat.com/articles/6329921 for details and instructions.\n",
                        "reason": "AdminAckRequired",
                        "status": "False",
                        "type": "Upgradeable"
                    }
                ],

In the above example, "Upgradeable=False" and "RetrievedUpdates=False" are set by CVO itself. BTW, this PR is also not mentioned RetrievedUpdates condition.

So this PR sounds like a bit confused for user.

[openshift-bot]

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

[openshift-ci]

@sagidlow: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[bergerhoffer]

The branch/enterprise-4.11 label has been added to this PR.

This is because your PR targets the main branch and is labeled for enterprise-4.10. And any PR going into main must also target the latest version branch (enterprise-4.11).

If the update in your PR does NOT apply to version 4.11 onward, please retarget this PR to go directly into the appropriate version branch or branches (enterprise-4.x) instead of main.

[openshift-bot]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[openshift-bot]

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

[openshift-ci]

@openshift-bot: Closed this PR.

Details

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue by commenting /reopen.
Mark the issue as fresh by commenting /remove-lifecycle rotten.
Exclude this issue from closing again by commenting /lifecycle frozen.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.