Skip to content

Conversation

@jottofar
Copy link
Contributor

This PR backports "Bug 1978376: Add admin ack Upgradeable condition gate" and "pkg/cvo/upgradeable: Enable admin-ack logic". It also adds the 4.8 gate "ack-4.8-kube-122-api-removals-in-4.9" which blocks upgrades to 4.9 until an admin performs the required acknowledgement step.

The PR is composed of 3 commits the first 2 of which were created with cherry-pick:

$ git cherry-pick 519b466
$ git cherry-pick 5b5db7d

@jottofar jottofar changed the title Bug 1999092: Add aand enable admin ack Upgradeable condition gate Bug 1999092: Add and enable admin ack Upgradeable condition gate Aug 31, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 31, 2021

@jottofar: An error was encountered querying GitHub for users with public email ([email protected]) for bug 1999092 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message. non-200 OK status code: 403 Forbidden body: "{\n \"documentation_url\": \"https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits\",\n \"message\": \"You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\"\n}\n"

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

Details

In response to this:

Bug 1999092: Add aand enable admin ack Upgradeable condition gate

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 31, 2021
data:
ack-4.8-kube-122-api-removals-in-4.9: |
Kubernetes 1.22 and thus OpenShift 4.9 remove several APIs which require admin consideration. Please see
https://docs.openshift.com/container-platform/4.8/release_notes/ocp-4-8-release-notes.html#ocp-4-8-deprecated-removed-features.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i hope we get to the point where we can point to better documentation for what an admin needs to go investigate/check before we ship this.

we should probably not merge this PR until we have that in place, since this gate will immediately start showing up in 4.8.z clusters.

@bparees
Copy link
Contributor

bparees commented Aug 31, 2021

/hold

we need the proper openshift documentation for what an admin is expected to do prior to ugprading to 4.9 (how to check api usage, how to ACK the gate, etc) before this merges, because once this merges anyone who upgrades to 4.8.z is going to start seeing this non-upgradeable condition.

@bparees bparees added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 31, 2021
@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 12, 2021
@wking
Copy link
Member

wking commented Sep 13, 2021

/bugzilla refresh

@openshift-ci openshift-ci bot added the bugzilla/severity-urgent Referenced Bugzilla bug's severity is urgent for the branch this PR is targeting. label Sep 13, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 13, 2021

@wking: This pull request references Bugzilla bug 1999092, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.z) matches configured target release for branch (4.8.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1978376 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 1978376 targets the "4.9.0" release, which is one of the valid target releases: 4.9.0
  • bug has dependents

Requesting review from QA contact:
/cc @jianlinliu

Details

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Sep 13, 2021
@openshift-ci openshift-ci bot requested a review from jianlinliu September 13, 2021 22:57
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 14, 2021
@jottofar
Copy link
Contributor Author

/retest

@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 24, 2021
jottofar and others added 2 commits September 24, 2021 10:02
519b466 (Bug 1978376: Add admin ack Upgradeable condition gate,
2021-07-27, openshift#633) had these commented out, because 4.9 has no built-in
acks.  But with the code commented out, it's hard to verify that the
logic works in 4.9 before backporting to 4.8 [1].  Enabling these
checks should be a no-op outside of verification, because admins are
unlikely to inject additional keys in the openshift-config-managed
namespace's admin-gates ConfigMap.  And it allows us to verify the
logic in 4.9 and cook there with live code before approving the 4.8
backports.  It's also one less thing we might forget before enabling
new admin acks in future versions, like 4.10 or later.

[1]: https://bugzilla.redhat.com/show_bug.cgi?id=1978376#c19
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 24, 2021
@openshift-ci openshift-ci bot requested a review from bparees September 24, 2021 14:10
@openshift-ci openshift-ci bot requested review from sdodson and wking September 24, 2021 14:10
Copy link
Member

@wking wking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With cherry-pick-diff:

$ cherry-pick-diff origin/release-4.8..origin/pr/647 origin/release-4.9
fatal: bad revision 'origin/release-4.9^{/admin-gates. Add ack-4.8-kube-122-api-removals-in-4.9 gate}'
??? -> 9e21155d admin-gates: Add ack-4.8-kube-122-api-removals-in-4.9 gate
5b5db7db -> a5db8728 pkg/cvo/upgradeable: Enable admin-ack logic
519b4667 -> 61946347 Bug 1978376: Add admin ack Upgradeable condition gate
fatal: bad revision 'origin/release-4.9^{/admin-gates. Add ack-4.8-kube-122-api-removals-in-4.9 gate}'

5b5db7db -> a5db8728 pkg/cvo/upgradeable: Enable admin-ack logic
...unimportant context changes...
519b4667 -> 61946347 Bug 1978376: Add admin ack Upgradeable condition gate
...unimportant context changes, plus some *Operator vs. Operator changes because 4.8 doesn't have #603...
...and in 4.9, #637's etcd backup landed after #633's admin ack, while in 4.8 this admin ack is landing after #649's etcd backup...

The unique-to-4.8 9e21155 looks good to me.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 24, 2021
@openshift-ci openshift-ci bot removed the lgtm Indicates that a PR is ready to be merged. label Sep 24, 2021
@jottofar
Copy link
Contributor Author

/retest

@sdodson
Copy link
Member

sdodson commented Sep 27, 2021

/lgtm
String change only since last review, awaiting confirmation of publishing date.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 27, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 27, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jottofar, sdodson, wking

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:
  • OWNERS [jottofar,sdodson,wking]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@dhellmann
Copy link

[patch-manager] ⌛ This pull request was not picked by the patch manager for the current z-stream window and have to wait for the next window.

skipped for today

  • Score: 0.00
  • Reason: skipping because "do-not-merge/hold" label found

NOTE: This message was automatically generated, if you have questions please ask on #forum-release

@sdodson
Copy link
Member

sdodson commented Sep 28, 2021

/hold cancel

@openshift-ci openshift-ci bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Sep 28, 2021
@jottofar
Copy link
Contributor Author

/retest

@dhellmann dhellmann added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Sep 28, 2021
@dhellmann
Copy link

[patch-manager] 🚀 Approved for z-stream by score: 1.80

picked

@wking
Copy link
Member

wking commented Sep 28, 2021

etcdHighNumberOfLeaderChanges is unrelated.

/override ci/prow/e2e-agnostic-upgrade

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 28, 2021

@wking: Overrode contexts on behalf of wking: ci/prow/e2e-agnostic-upgrade

Details

In response to this:

etcdHighNumberOfLeaderChanges is unrelated.

/override ci/prow/e2e-agnostic-upgrade

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-merge-robot openshift-merge-robot merged commit 1b4c2ef into openshift:release-4.8 Sep 28, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 28, 2021

@jottofar: Some pull requests linked via external trackers have merged:

The following pull requests linked via external trackers have not merged:

These pull request must merge or be unlinked from the Bugzilla bug in order for it to move to the next state. Once unlinked, request a bug refresh with /bugzilla refresh.

Bugzilla bug 1999092 has not been moved to the MODIFIED state.

Details

In response to this:

Bug 1999092: Add and enable admin ack Upgradeable condition gate

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

wking added a commit to wking/openshift-release that referenced this pull request Sep 29, 2021
…e-1.22-api-removals-in-4.9

Avoid [1]:

  alert ClusterNotUpgradeable fired for 1115 seconds with labels: {condition="Upgradeable", endpoint="metrics", name="version", severity="warning"}

for tests that install 4.8 and then sit on it for over an hour, now
that [2] has landed in 4.8.  [3] will follow up with code to handle CI
that installs an earlier release and updates into 4.8 or later.

[1]: https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.8-e2e-aws-serial/1443018123379740672
[2]: openshift/cluster-version-operator#647
[3]: openshift/origin#26446
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-urgent Referenced Bugzilla bug's severity is urgent for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants