OCPBUGS-105207: Revert "OCPBUGS-89689: (karpenter) use completed release image for unpinned NodeClaims during CP upgrade"

[ironcladlou]

Reverts #8957

Justification: https://redhat.atlassian.net/browse/OCPBUGS-105207

PR #8957 (merged fff9961de5, Aug 4 11:10 UTC) introduced a new assertNodeClaimsNotDrifted test assertion in TestKarpenterUpgradeControlPlane. Before this PR, the test passed 13 consecutive runs. After the merge, 3 of 7 runs failed, all triggered by the new assertion.

This impacts the blocking merge path as well as the release controller.

Summary by CodeRabbit

• New Features

  • Improved control plane upgrades by using the configured release image directly.
  • Node configuration versions now track the most recent completed cluster version, with safer fallback behavior during initial rollout.

• Bug Fixes

  • Improved handling when version information is temporarily unavailable.
  • Node configuration drift is now detected alongside control plane image rollouts, enabling updated nodes to be recognized more reliably.

• Tests

  • Expanded upgrade coverage for completed, partial, initial, and unavailable version states.

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[openshift-ci-robot]

@ironcladlou: This pull request references Jira Issue OCPBUGS-105207, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Reverts #8957

Justification: https://redhat.atlassian.net/browse/OCPBUGS-105207

PR #8957 (merged fff9961de5, Aug 4 11:10 UTC) introduced a new assertNodeClaimsNotDrifted test assertion in TestKarpenterUpgradeControlPlane. Before this PR, the test passed 13 consecutive runs. After the merge, 3 of 7 runs failed, all triggered by the new assertion.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Note

Reviews paused

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The controller now uses the HostedControlPlane release image and computes the NodeClass version from cluster version status. Tests cover completed history, partial history, desired-version fallback, and unavailable status. The control plane upgrade test now observes NodeClaim drift concurrently with the image rollout. The previous background no-drift polling helper was removed.

Sequence Diagram(s)

sequenceDiagram
  participant UpgradeTest
  participant HostedCluster
  participant ControlPlane
  participant NodeClaims
  UpgradeTest->>ControlPlane: Start image rollout
  ControlPlane->>HostedCluster: Update rollout state
  UpgradeTest->>NodeClaims: Wait for initial claims to drift
  NodeClaims-->>UpgradeTest: Report drift completion
  UpgradeTest->>HostedCluster: Refresh and continue

Loading

Suggested reviewers: maxcao13, judexzhu

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Test Structure And Quality	❓ Inconclusive	Investigation is still in progress.	Inspect the changed e2e test, helper timeouts, cleanup, and repository patterns before deciding.

✅ Passed checks (10 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	Changed tests use testing.T, not Ginkgo; all test/subtest titles are static descriptive strings, and dynamic resource values remain in body assertion messages.
Topology-Aware Scheduling Compatibility	✅ Passed	The patch changes release-image/version resolution and drift-test timing only; added lines introduce no affinity, topology spread, node selector, toleration, replica, or PDB constraints.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	The patch adds no Ginkgo e2e declarations. Its e2e additions only wait for NodeClaim drift and add no IPv4 literals or external connectivity.
No-Weak-Crypto	✅ Passed	The four modified files add no crypto imports, weak algorithms, cipher modes, custom crypto, or secret/token comparisons; the diff only changes version reconciliation and test timing.
Container-Privileges	✅ Passed	The patch changes only four Go files; no manifest changed, and scans found none of the listed privilege settings in the changed files.
No-Sensitive-Data-In-Logs	✅ Passed	The commit adds no logging calls and no sensitive literals; its only added runtime data flow uses releaseImage and version without emitting them to logs.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly identifies the pull request as a revert of the completed release image change, which matches the code and test changes.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-ci-robot]

@ironcladlou: This pull request references Jira Issue OCPBUGS-105207, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (5.0.0) matches configured target version for branch (5.0.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Details

In response to this:

Reverts #8957

Justification: https://redhat.atlassian.net/browse/OCPBUGS-105207

PR #8957 (merged fff9961de5, Aug 4 11:10 UTC) introduced a new assertNodeClaimsNotDrifted test assertion in TestKarpenterUpgradeControlPlane. Before this PR, the test passed 13 consecutive runs. After the merge, 3 of 7 runs failed, all triggered by the new assertion.

Summary by CodeRabbit

• New Features

• Improved control plane upgrades by using the configured release image directly.

• Node configuration versions now track the most recent completed cluster version, with safer fallback behavior during initial rollout.

• Bug Fixes

• Improved handling when version information is temporarily unavailable.

• Node configuration drift is now detected alongside control plane image rollouts, enabling updated nodes to be recognized more reliably.

• Tests

• Expanded upgrade coverage for completed, partial, initial, and unavailable version states.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[coderabbitai]

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

karpenter-operator/controllers/karpenterignition/karpenterignition_controller.go (1)

345-380: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Make the fallback contract match the implementation.

currentClusterVersion returns Desired.Version whenever no completed entry exists. This includes empty and multi-entry histories. The comment limits the fallback to one entry, and the len(...) == 1 branch has the same result as the final return.

Proposed cleanup

-// and the most recent CompletionTime. If no completed entries exist and there is exactly one
-// history entry, it falls back to the desired version. This handles the case where a cluster
-// is still rolling out its initial version.
+// and the most recent CompletionTime. If no completed entries exist, it falls
+// back to the desired version.
 ...
-	// If there are no completed entries but exactly one history entry exists, the cluster
-	// is likely still rolling out its first version. Fall back to the desired version.
-	if len(hostedCluster.Status.Version.History) == 1 {
-		return hostedCluster.Status.Version.Desired.Version
-	}
-
 	return hostedCluster.Status.Version.Desired.Version

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@karpenter-operator/controllers/karpenterignition/karpenterignition_controller.go`
around lines 345 - 380, Update currentClusterVersion so its comments and control
flow reflect that Desired.Version is returned whenever no completed history
entry exists. Remove the redundant len(History) == 1 conditional and consolidate
the fallback into the final return, while preserving the completed-entry
selection logic.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/karpenter_control_plane_upgrade_test.go`:
- Around line 84-96: Update the concurrent drift verification around the
goroutine and e2eutil.WaitForImageRollout so rollout completion is explicitly
signaled before waiting for drift results. Have the NodeClaim drift checks
record or fail when the first NodeClaim reaches Drifted=True after that
completion signal, while preserving the existing wait for all initial NodeClaims
to drift.

---

Outside diff comments:
In
`@karpenter-operator/controllers/karpenterignition/karpenterignition_controller.go`:
- Around line 345-380: Update currentClusterVersion so its comments and control
flow reflect that Desired.Version is returned whenever no completed history
entry exists. Remove the redundant len(History) == 1 conditional and consolidate
the fallback into the final return, while preserving the completed-entry
selection logic.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 738bea17-72f5-4640-be91-5b71c859d5c4

📥 Commits

Reviewing files that changed from the base of the PR and between 8317d4c and bc3d0a6.

📒 Files selected for processing (4)

• karpenter-operator/controllers/karpenterignition/karpenterignition_controller.go
• karpenter-operator/controllers/karpenterignition/karpenterignition_controller_test.go
• test/e2e/karpenter_control_plane_upgrade_test.go
• test/e2e/karpenter_test.go

💤 Files with no reviewable changes (1)

• test/e2e/karpenter_test.go

[ironcladlou]

@coderabbitai pause

[coderabbitai]

✅ Action performed

Reviews paused.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 44.96%. Comparing base (8317d4c) to head (bc3d0a6).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9233   +/-   ##
=======================================
  Coverage   44.96%   44.96%           
=======================================
  Files         778      778           
  Lines       97452    97448    -4     
=======================================
+ Hits        43820    43821    +1     
+ Misses      50607    50603    -4     
+ Partials     3025     3024    -1     

Files with missing lines	Coverage Δ
.../karpenterignition/karpenterignition_controller.go	64.69% <100.00%> (+0.70%)	⬆️

Flag	Coverage Δ
cmd-support	38.62% <ø> (ø)
cpo-hostedcontrolplane	47.28% <ø> (ø)
cpo-other	45.67% <ø> (ø)
hypershift-operator	55.00% <ø> (ø)
other	34.32% <100.00%> (+0.02%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bryan-cox]

/approve
/lgtm
/verified bypass

[openshift-ci-robot]

@bryan-cox: The verified label has been added.

Details

In response to this:

/approve
/lgtm
/verified bypass

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-azure-self-managed
/test e2e-v2-gke

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox, ironcladlou

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [bryan-cox]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[cwbotbot]

Test Results

e2e-aws

• Status: ✅ PASS
• Started: 2026-08-06T00:49:17Z
• View Job
• View Job History

e2e-aks

• Status: ✅ PASS
• Started: 2026-08-06T00:49:17Z
• View Job
• View Job History

[openshift-merge-bot]

/retest-required

Remaining retests: 0 against base HEAD 96b041f and 2 for PR HEAD bc3d0a6 in total

[openshift-ci]

@ironcladlou: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci-robot]

@ironcladlou: Jira Issue Verification Checks: Jira Issue OCPBUGS-105207
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-105207 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

Reverts #8957

Justification: https://redhat.atlassian.net/browse/OCPBUGS-105207

PR #8957 (merged fff9961de5, Aug 4 11:10 UTC) introduced a new assertNodeClaimsNotDrifted test assertion in TestKarpenterUpgradeControlPlane. Before this PR, the test passed 13 consecutive runs. After the merge, 3 of 7 runs failed, all triggered by the new assertion.

This impacts the blocking merge path as well as the release controller.

Summary by CodeRabbit

• New Features

• Improved control plane upgrades by using the configured release image directly.

• Node configuration versions now track the most recent completed cluster version, with safer fallback behavior during initial rollout.

• Bug Fixes

• Improved handling when version information is temporarily unavailable.

• Node configuration drift is now detected alongside control plane image rollouts, enabling updated nodes to be recognized more reliably.

• Tests

• Expanded upgrade coverage for completed, partial, initial, and unavailable version states.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-merge-robot]

Fix included in release 5.0.0-0.nightly-2026-08-06-203809