Skip to content

Conversation

@guillaumerose
Copy link
Contributor

@guillaumerose guillaumerose commented Dec 10, 2020

Using the installer, if a cluster operator is in overrides list, it is
still created during the precreation step.
When both the cluster operator and the deployment of an operator are
overridden, the CVO (and the installer) waits for ever the end of the
end of the provisioning.


I faced this when I tried to disable monitoring for CRC

I1210 20:28:04.418715       1 sync_worker.go:687] Precreated resource clusteroperator "monitoring" (317 of 617)
I1210 20:28:32.633406       1 sync_worker.go:701] Running sync for clusteroperator "monitoring" (317 of 617)
I1210 20:28:32.633423       1 sync_worker.go:705] Skipping clusteroperator "monitoring" (317 of 617) as unmanaged

It means that we can't disable fully an operator with overrides. It has to be deployed once.

…in overrides

Using the installer, if a cluster operator is in overrides list, it is
still created during the precreation step.
When both the cluster operator and the deployment of an operator are
overridden, the CVO (and the installer) waits for ever the end of the
end of the provisioning.
@guillaumerose guillaumerose changed the title Don't create ClusterOperator during precreation step if it's present in overrides Bug 1907313: Don't create ClusterOperator during precreation step if it's present in overrides Dec 14, 2020
@openshift-ci-robot openshift-ci-robot added bugzilla/severity-unspecified Referenced Bugzilla bug's severity is unspecified for the PR. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Dec 14, 2020
@openshift-ci-robot
Copy link
Contributor

@guillaumerose: This pull request references Bugzilla bug 1907313, which is invalid:

  • expected the bug to target the "4.7.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

Bug 1907313: Don't create ClusterOperator during precreation step if it's present in overrides

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@LalatenduMohanty
Copy link
Member

/bugzilla refresh

@openshift-ci-robot openshift-ci-robot added bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/severity-unspecified Referenced Bugzilla bug's severity is unspecified for the PR. labels Dec 14, 2020
@openshift-ci-robot
Copy link
Contributor

@LalatenduMohanty: This pull request references Bugzilla bug 1907313, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Dec 14, 2020
if task.Manifest.GVK != configv1.SchemeGroupVersion.WithKind("ClusterOperator") {
continue
}
ov, ok := getOverrideForManifest(work.Overrides, task.Manifest)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't be this check come before the if task.Manifest.GVK != configv1.SchemeGroupVersion.WithKind("ClusterOperator") ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could, it is equivalent.

The only difference will be that CVO will log for all types of objects, like: Skipping precreation of deployment foobar as unmanaged. It can be misleading as we only precreate ClusterOperator and not others objects.

@wking
Copy link
Member

wking commented Dec 14, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Dec 14, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: guillaumerose, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 14, 2020
@guillaumerose
Copy link
Contributor Author

/retest

2 similar comments
@guillaumerose
Copy link
Contributor Author

/retest

@guillaumerose
Copy link
Contributor Author

/retest

@openshift-merge-robot openshift-merge-robot merged commit b2392e9 into openshift:master Dec 15, 2020
@openshift-ci-robot
Copy link
Contributor

@guillaumerose: All pull requests linked via external trackers have merged:

Bugzilla bug 1907313 has been moved to the MODIFIED state.

Details

In response to this:

Bug 1907313: Don't create ClusterOperator during precreation step if it's present in overrides

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking added a commit to wking/cluster-version-operator that referenced this pull request Mar 28, 2021
This is a hack fix for [1], where we have a delay on 4.6->4.7 updates,
and on some 4.7 installs, between the very early ClusterOperator
precreation and the operator eventually coming up to set its status
conditions.  In the interim, there are no conditions, which causes
cluster_operator_up to be 0, which causes the critical
ClusterOperatorDown to fire.  We'll want a more general fix going
forward, this commit is a temporary hack to avoid firing the critical
ClusterOperatorDown while we build consensus around the general fix.

The downside to dropping precreates for this operator is that we lose
the must-gather references when the operator fails to come up.  That
was what precreation was designed to address in 2a469e3 (cvo: When
installing or upgrading, fast-fill cluster-operators, 2020-02-07, openshift#318).
If we actually get a must-gather without the bare-metal bits and we
miss them, we can revisit the approach this hack is taking.

Manually picked back to 4.6, which doesn't include b0f73af (Don't
create ClusterOperator during precreation step if it's present in
overrides, 2020-12-10, openshift#488).

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1929917
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-low Referenced Bugzilla bug's severity is low for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants