hotfix: skip z-stream upgrade test for 4.19 - #4621
Conversation
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: raelga The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Rael Garcia (raelga)
changed the title
There was a problem hiding this comment.
Pull request overview
Disables the 4.19 control-plane “candidate channel” z-stream upgrade E2E entry to avoid failures caused by Cincinnati returning versions not served by the RP, until version filtering lands (referenced PR #4618).
Changes:
- Commented out the Ginkgo
Entryfor version 4.19 in the z-stream upgradeDescribeTable. - Added a TODO with a link to PR #4618 as the re-enable condition.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // TODO: re-enable once https://github.com/Azure/ARO-HCP/pull/4618 is merged | ||
| // Entry("for 4.19", labels.RequireNothing, labels.Critical, labels.Positive, labels.AroRpApiCompatible, "4.19"), |
There was a problem hiding this comment.
Commenting out the 4.19 Entry removes it entirely from Ginkgo reporting, making it easy to forget and harder to track in CI. Consider keeping the spec present but explicitly skipped/pending (e.g., a Skip at the very start of the table function when version == "4.19", ideally with a timebomb deadline like other e2e tests) so the disablement is visible and self-expiring.
ba0cc5b to
bfdc578
Compare
The 4.19 z-stream upgrade test fails because Cincinnati returns versions that the RP does not serve. Disable the 4.19 entry until version filtering is implemented.
bfdc578 to
4a42871
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // TODO: remove once https://github.com/Azure/ARO-HCP/pull/4618 is merged | ||
| if version == "4.19" && time.Now().Before(time.Date(2026, 4, 1, 0, 0, 0, 0, time.UTC)) { | ||
| Skip("4.19 z-stream test temporarily skipped: Cincinnati returns versions the RP doesn't serve") | ||
| } |
There was a problem hiding this comment.
Can we skip it here
It can happen that even for 4.20+ versions, a new version which just got release is selected just before it's loaded in CS database by the worker. In this case, you'll still get
{
"error": {
"code": "InvalidRequestContent",
"message": "Version 'openshift-v4.y.z-candidate' doesn't exist"
}
}
So I'll skip the test based on that error up for now to avoid making our test fail that run. Chances of it happening are pretty low but can happen. That will go away when we move the creation of the cluster to the backend - being discussed here #4477
There was a problem hiding this comment.
This can be a followup
There was a problem hiding this comment.
Manyanda Chitimbo (@machi1990) How will moving the cluster creation call solve the issue of a new version selected before getting loaded in CS?
There was a problem hiding this comment.
Jakob Gray (@JakobGray) good question.
Because the create cluster controller will be the one to retry the operation and not the test.
CS already loads all the version in an interval of 5 minutes. When a new one is available, cs discovers it in that worker and loads it (See the maxVersion entries in the config ).
The idea is that the controller will do this:
-
call CS with the 4.Y.Z
-
What could happen then is:
- CS doesn't requested image in its DB because not reconciled yet; CS returns the error above
- The controller requeues on that for the call to be retried later
- Eventually (in 5minutes) cs discovers the new image the call is accepted
How does that sound with you?
There was a problem hiding this comment.
I can see that for cluster create, but for an automatic z-stream upgrade don't we always run the risk of trying to upgrade before CS has discovered the new image?
There was a problem hiding this comment.
Z-stream upgrade triggers are also retriable in the same sense. When a new x.y.z has been computed; a call is made. If CS errors out - because an upgrade graph is missing on CS' database, it'll be retried.
|
This is blocking all PRs. |
The 4.19 z-stream upgrade test fails because Cincinnati returns versions that the RP does not serve. Disable the 4.19 entry until version filtering is implemented or the 1 week time bomb expires.
Revert once #4618 fixes the issue.