OTA-1223: config/v1/types_cluster_version: Drop enum from ClusterVersion status capabilities #1624
Conversation
|
Hello @wking! Some important instructions when contributing to openshift/api: |
openshift-ci
bot
added
the
size/XS
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: wking The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
… capabilities
The cluster-version operator should be the only actor writing to
ClusterVersion status. Trust it to pick appropriate values, instead
of restricting it with an enum.
When updating into new capabilities, the current timeline is:
1. Outgoing CVO accepts the new target, and figures out verified and
implicit capabilities.
2. Outoing CVO starts trying to write to status, but because of the
new capability not being a part of the outgoing CRD's capability
enum, this fails.
3. Outgoing CVO pushes the incoming ClusterVersion CRD in runlevel 0
index 1 [1].
4. status sync attempts, if any, start working again.
5. Outgoing CVO pushes the incoming CVO Deployment in runlevel 0 index
3 [2].
6. Deployment controller TERMs the outgoing CVO process.
7. Outgoing CVO wraps up the manifest sycning.
8. Outgoing CVO attempts a final status sync [3].
9. Outgoing CVO releases the leader lock [4].
10. Outgoing CVO exists.
With the status enum removal in this commit, ClusterVersion status
syncing will remain working the whole time, reducing the risk of the
outgoing CVO shutting down before it had recorded important
information in status.
The enum relaxation will also help with rollbacks that drop
capabilities. Those currently fail on the same status-sync issue [5]:
$ oc -n openshift-cluster-version logs -l k8s-app=cluster-version-operator --tail 50 | grep 'ClusterVersion.config.openshift.io "version" is invalid:' | tail -n1
I1005 01:41:28.165931 1 cvo.go:600] Dropping "openshift-cluster-version/version" out of the queue &{0xc00039eae0 0xc0004142e8}: ClusterVersion.config.openshift.io "version" is invalid: [status.capabilities.knownCapabilities[0]: Unsupported value: "Build": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[3]: Unsupported value: "DeploymentConfig": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[4]: Unsupported value: "ImageRegistry": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[6]: Unsupported value: "MachineAPI": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[0]: Unsupported value: "Build": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[3]: Unsupported value: "DeploymentConfig": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[4]: Unsupported value: "ImageRegistry": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[6]: Unsupported value: "MachineAPI": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning"]
but unlike the roll-forward case, where that is a temporary issue, on
rollbacks the additional capabilities never became acceptable again,
and the cluster-version operator was permanently blocked from writing
to ClusterVersion status. With the status enum removal in this
commit, ClusterVersion status syncing will remain working, even though
it will list as an enabled capability the capability which had been
added during the earlier roll-forward.
[1]: https://github.com/openshift/cluster-version-operator/blob/baf7ba7b45852ad2b95e9e498585fbee35111eef/Dockerfile.rhel#L11
[2]: https://github.com/openshift/cluster-version-operator/blob/baf7ba7b45852ad2b95e9e498585fbee35111eef/install/0000_00_cluster-version-operator_03_deployment.yaml
[3]: https://github.com/openshift/cluster-version-operator/blob/baf7ba7b45852ad2b95e9e498585fbee35111eef/pkg/cvo/cvo.go#L421-L423
[4]: https://github.com/openshift/cluster-version-operator/blob/baf7ba7b45852ad2b95e9e498585fbee35111eef/pkg/start/start.go#L258-L278
[5]: openshift/release#43984 (comment)
wking
force-pushed
the
drop-status-capability-enum
branch
from
c475e7a to
75cddc1
Compare
openshift-ci
bot
added
size/M
openshift-merge-robot
added
the
needs-rebase
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
wking
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@wking: This pull request references OTA-1223 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.16.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
+1 to this from a hive (fleet manager) perspective. Scenario:
|
|
Issues go stale after 90d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle stale |
openshift-ci
bot
added
the
lifecycle/stale
|
Stale issues rot after 30d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle rotten |
openshift-ci
bot
added
lifecycle/rotten
|
@wking: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
bump |
|
Rotten issues close after 30d of inactivity. Reopen the issue by commenting /close |
|
@openshift-bot: Closed this PR. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
6 participants
The cluster-version operator should be the only actor writing to ClusterVersion status. Trust it to pick appropriate values, instead of restricting it with an enum.
When updating into new capabilities, the current timeline is:
With the status enum removal in this commit, ClusterVersion status syncing will remain working the whole time, reducing the risk of the outgoing CVO shutting down before it had recorded important information in status.
The enum relaxation will also help with rollbacks that drop capabilities. Those currently fail on the same status-sync issue:
$ oc -n openshift-cluster-version logs -l k8s-app=cluster-version-operator --tail 50 | grep 'ClusterVersion.config.openshift.io "version" is invalid:' | tail -n1 I1005 01:41:28.165931 1 cvo.go:600] Dropping "openshift-cluster-version/version" out of the queue &{0xc00039eae0 0xc0004142e8}: ClusterVersion.config.openshift.io "version" is invalid: [status.capabilities.knownCapabilities[0]: Unsupported value: "Build": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[3]: Unsupported value: "DeploymentConfig": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[4]: Unsupported value: "ImageRegistry": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.knownCapabilities[6]: Unsupported value: "MachineAPI": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[0]: Unsupported value: "Build": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[3]: Unsupported value: "DeploymentConfig": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[4]: Unsupported value: "ImageRegistry": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning", status.capabilities.enabledCapabilities[6]: Unsupported value: "MachineAPI": supported values: "openshift-samples", "baremetal", "marketplace", "Console", "Insights", "Storage", "CSISnapshot", "NodeTuning"]but unlike the roll-forward case, where that is a temporary issue, on rollbacks the additional capabilities never became acceptable again, and the cluster-version operator was permanently blocked from writing to ClusterVersion status. With the status enum removal in this commit, ClusterVersion status syncing will remain working, even though it will list as an enabled capability the capability which had been added during the earlier roll-forward.