Skip to content

Bug 2028930: test/extended/operators: Allow early KubeletMinorVersionUnsupportedNextUpgrade #26531

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
openshift-merge-robot merged 2 commits into from
Oct 21, 2021
Merged

Bug 2028930: test/extended/operators: Allow early KubeletMinorVersionUnsupportedNextUpgrade #26531

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ddeb3ce
test/extended/operators: Guess at Upgradeable exception removal
wking Oct 20, 2021
619200e
test/extended/operators: Allow early KubeletMinorVersionUnsupportedNe…
wking Oct 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 22 additions & 4 deletions test/extended/operators/operators.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -235,15 +235,33 @@ func surprisingConditions(co objx.Map) ([]configv1.ClusterOperatorStatusConditio
expected = configv1.ConditionTrue
}
if cond.Get("status").String() != string(expected) {
if conditionType == configv1.OperatorUpgradeable && (name == "kube-storage-version-migrator" || // https://bugzilla.redhat.com/show_bug.cgi?id=1928141
name == "openshift-controller-manager" || // https://bugzilla.redhat.com/show_bug.cgi?id=1948011
name == "service-ca") { // https://bugzilla.redhat.com/show_bug.cgi?id=1948012
reason := cond.Get("reason").String()
if conditionType == configv1.OperatorUpgradeable && (name == "kube-storage-version-migrator" || // https://bugzilla.redhat.com/show_bug.cgi?id=1928141 , currently fixed for 4.10, but no backports at the moment. We currently have ...-upgrade-4.y-to-4.(y+1)-to-4.(y+2)-to-4.(y+3)-ci jobs, so as long as we don't extend that +3 skew for those jobs, we should be able to drop this code once 4.13 forks off of the development branch.
name == "openshift-controller-manager" || // https://bugzilla.redhat.com/show_bug.cgi?id=1948011 , currently fixed for 4.8, but no backports at the moment. We currently have ...-upgrade-4.y-to-4.(y+1)-to-4.(y+2)-to-4.(y+3)-ci jobs, so as long as we don't extend that +3 skew for those jobs, we should be able to drop this code once 4.10 forks off the development branch.
name == "service-ca" || // https://bugzilla.redhat.com/show_bug.cgi?id=1948012 , currently fixed for 4.8, but no backports at the moment. We currently have ...-upgrade-4.y-to-4.(y+1)-to-4.(y+2)-to-4.(y+3)-ci jobs, so as long as we don't extend that +3 skew for those jobs, we should be able to drop this code once 4.10 forks off the development branch.

// Allow some node skew for post-update unpause monitoring. Without this,
// 4.8->4.9->4.10 jobs are updating successfully to 4.10, unpausing
// compute, running the post-unpause compute-settling monitor, and
// failing on [1,2]:
//
// : [sig-arch][Early] Managed cluster should start all core operators [Skipped:Disconnected] [Suite:openshift/conformance/parallel] 0s
// fail [github.com/onsi/[email protected]+incompatible/internal/leafnodes/runner.go:113]: Oct 17 23:28:57.284: Some cluster operators are not ready: kube-apiserver (Upgradeable=False KubeletMinorVersion_KubeletMinorVersionUnsupportedNextUpgrade: KubeletMinorVersionUpgradeable: Kubelet minor versions on nodes ip-10-0-135-91.ec2.internal, ip-10-0-168-151.ec2.internal, and ip-10-0-192-244.ec2.internal will not be supported in the next OpenShift minor version upgrade.)
//
// With this change, that skew guard from [3] is allowed to trip early in
// the suite. But if it trips for long enough to set off alerts, we'd
// still fail on that.
//
// [1]: https://testgrid.k8s.io/redhat-openshift-ocp-release-4.10-informing#periodic-ci-openshift-release-master-nightly-4.10-upgrade-from-stable-4.8-e2e-aws-upgrade-paused
// [2]: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.10-upgrade-from-stable-4.8-e2e-aws-upgrade-paused/1449821870344900608
// [3]: https://github.com/openshift/cluster-kube-apiserver-operator/pull/1199
(name == "kube-apiserver" && reason == "KubeletMinorVersion_KubeletMinorVersionUnsupportedNextUpgrade")) {
continue
}
badConditions = append(badConditions, configv1.ClusterOperatorStatusCondition{
Type: conditionType,
Status: configv1.ConditionStatus(cond.Get("status").String()),
Reason: cond.Get("reason").String(),
Reason: reason,
Message: cond.Get("message").String(),
})
}
Expand Down