CNTRLPLANE-3632: Predictable NodePool rollout control

[csrwng]

What this PR does / why we need it:

Decouples NodePool rollout triggering from management-side configuration changes (e.g. HAProxy image digest bumps) so that only user-driven spec changes cause worker node replacement.

Today the NodePool controller uses a single hash over the entire rendered ignition config — including management-side image references — to drive rollout decisions. Any change to this hash triggers a full Replace/InPlace rollout. This means automated HAProxy image updates (which happen on every HyperShift operator upgrade) cause unnecessary node churn.

This PR introduces:

• RolloutHash() / RolloutHashWithoutVersion(): New hash methods that include only spec-driven inputs (user MachineConfigs, release version, pull secret, trust bundle, global config), excluding management-side content like HAProxy
• nodePoolCurrentRolloutConfig annotation: Tracks the current rollout config hash, used for rollout decisions instead of comparing data secret names
• Annotation seeding: On first reconcile after operator upgrade, the annotation is populated without triggering a rollout
• isUpdatingConfig() safety: Returns false when the annotation is absent to prevent condition flip-flop during upgrade

The existing Hash() and payload secret naming are unchanged — new nodes always get the latest payload including management-side content when they ARE replaced for spec-driven reasons.

Which issue(s) this PR fixes:

Fixes OCPSTRAT-3298

Special notes for your reviewer:

• RolloutHashWithoutVersion() intentionally includes globalConfig unlike the existing HashWithoutVersion() which omits it. Proxy/image config changes should trigger rollouts.
• The parse() → doParse() refactor is necessary because haproxy content is prepended inside parse() and cannot be stripped from the final string after the fact.
• The secret_janitor_test.go build error is pre-existing and unrelated to this PR.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

Release Notes

• New Features

  • Added rollout configuration tracking for NodePool updates, including automatic initialization after operator upgrades.
  • Added a condition indicating configuration changes are pending until the next spec-driven rollout.

• Bug Fixes

  • Management-side HAProxy and image changes no longer trigger unnecessary node replacements.
  • Rollouts now respond predictably to meaningful configuration, version, and template changes.
  • Improved token handling prevents unnecessary secret recreation.
  • Configuration and rollout status remain synchronized, avoiding no-op updates that could delay completion tracking.

[openshift-ci-robot]

@csrwng: This pull request references OCPSTRAT-3298 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the feature to target either version "5.0." or "openshift-5.0.", but it targets "openshift-5.1" instead.

Details

In response to this:

What this PR does / why we need it:

Decouples NodePool rollout triggering from management-side configuration changes (e.g. HAProxy image digest bumps) so that only user-driven spec changes cause worker node replacement.

Today the NodePool controller uses a single hash over the entire rendered ignition config — including management-side image references — to drive rollout decisions. Any change to this hash triggers a full Replace/InPlace rollout. This means automated HAProxy image updates (which happen on every HyperShift operator upgrade) cause unnecessary node churn.

This PR introduces:

• RolloutHash() / RolloutHashWithoutVersion(): New hash methods that include only spec-driven inputs (user MachineConfigs, release version, pull secret, trust bundle, global config), excluding management-side content like HAProxy
• nodePoolCurrentRolloutConfig annotation: Tracks the current rollout config hash, used for rollout decisions instead of comparing data secret names
• Annotation seeding: On first reconcile after operator upgrade, the annotation is populated without triggering a rollout
• isUpdatingConfig() safety: Returns false when the annotation is absent to prevent condition flip-flop during upgrade

The existing Hash() and payload secret naming are unchanged — new nodes always get the latest payload including management-side content when they ARE replaced for spec-driven reasons.

Which issue(s) this PR fixes:

Fixes OCPSTRAT-3298

Special notes for your reviewer:

• RolloutHashWithoutVersion() intentionally includes globalConfig unlike the existing HashWithoutVersion() which omits it. Proxy/image config changes should trigger rollouts.
• The parse() → doParse() refactor is necessary because haproxy content is prepended inside parse() and cannot be stripped from the final string after the fact.
• The secret_janitor_test.go build error is pre-existing and unrelated to this PR.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

This PR separates full configuration hashes from rollout-specific hashes. Rollout hashes exclude management-side HAProxy content and platform-derived defaults. NodePool reconciliation stores the rollout hash in a dedicated annotation and reports management-side drift separately. MachineDeployment and MachineSet updates compare rollout hashes and versions, then record rollout completion conditionally. Token and Karpenter reconciliation track both hash types. Unit and E2E tests cover stable management changes, annotation seeding, and spec-driven rollouts.

Sequence Diagram(s)

sequenceDiagram
  participant NodePool
  participant Controller
  participant ConfigGenerator
  participant MachineDeployment
  participant MachineSet
  ConfigGenerator->>Controller: return full and rollout hashes
  Controller->>NodePool: seed or read rollout annotation
  Controller->>MachineDeployment: compare version and rollout hash
  Controller->>MachineSet: compare version and rollout hash
  MachineDeployment-->>Controller: report rollout completion
  MachineSet-->>Controller: report rollout completion
  Controller->>NodePool: persist current rollout state

Loading

Possibly related PRs

• openshift/hypershift#9136: Changes global configuration inputs used by NodePool hash computation.

🚥 Pre-merge checks | ✅ 10 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Test Structure And Quality	⚠️ Warning	The suite uses timed waits, but many cluster Get/List assertions lack failure messages, and the upgrade test does not reliably restore the removed rollout annotation.	Add diagnostic messages to every cluster assertion. Register cleanup before mutation and restore the original rollout and HAProxy annotations, including failure paths.

✅ Passed checks (10 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the primary change: predictable control of NodePool rollouts.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names	✅ Passed	All added It and When titles are literal, descriptive strings; no generated names, timestamps, namespaces, nodes, IPs, UUIDs, or computed values appear in test titles.
Topology-Aware Scheduling Compatibility	✅ Passed	The PR changes rollout hashing, annotations, CAPI machine specs, token state, and tests; it adds no affinity, topology spread, node selectors, tolerations, replica derivation, or PDB constraints.
Ipv6 And Disconnected Network Test Compatibility	✅ Passed	New Ginkgo tests use cluster API clients and node lists only; they contain no IPv4 logic, network resources, URL calls, or external downloads. The quay.io value is only an HAProxy annotation string.
No-Weak-Crypto	✅ Passed	PR additions use existing FNV-1a HashSimple for rollout identifiers; no MD5, SHA-1, DES, RC4, Blowfish, ECB, custom crypto, or secret/token comparisons were introduced.
Container-Privileges	✅ Passed	The PR changes only Go files; no added manifest or workload security settings match privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or allowPrivilegeEscalation.
No-Sensitive-Data-In-Logs	✅ Passed	New logging emits rollout hashes or fixed status text; no passwords, tokens, API keys, PII, or payload data. The releaseImage log was already present before this PR.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[openshift-merge-bot]

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

[coderabbitai]

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@hypershift-operator/controllers/nodepool/config_test.go`:
- Around line 742-794: Replace the manual annotation writes and plain equality
checks in TestRolloutHashAnnotationSeeding with calls into the real production
seeding helper (the function that computes/returns the rollout hash and a seeded
bool) and assert the boolean return; specifically, when
nodePoolAnnotationCurrentRolloutConfig is absent call the seeding helper and
expect seeded==true and the returned hash to be stored on nodePool.Annotations,
and when the annotation already exists call the helper and expect seeded==false;
also update TestGenerateMCORawConfig to consume and assert on rolloutConfigsRaw
(rather than discarding it) so changes to the generated rollout-only config
(e.g., reintroducing HAProxy) will fail tests.

In `@hypershift-operator/controllers/nodepool/config.go`:
- Around line 155-192: generateMCORawConfig currently only strips inline
cg.haproxyRawConfig via parseWithoutHaproxy, but when cg.haproxyRawConfig == ""
getCoreConfigs still returns the legacy HAProxy core ConfigMap which then ends
up in rolloutConfigsRaw; change generateMCORawConfig so that before calling
parseWithoutHaproxy it builds a filtered configs slice that excludes the legacy
HAProxy core ConfigMap (the same ConfigMap returned by getCoreConfigs for
HAProxy — detect it by the same unique identifier used in getCoreConfigs, e.g.,
its ConfigMap name/label or by comparing its data to cg.haproxyRawConfig), then
pass that filtered slice to parseWithoutHaproxy while still passing the full
configs slice to parse so fullConfig is unchanged (use functions/fields:
generateMCORawConfig, getCoreConfigs, getUserConfigs, getNTOGeneratedConfig,
cg.haproxyRawConfig, parse, parseWithoutHaproxy).

In `@hypershift-operator/controllers/nodepool/nodepool_controller.go`:
- Around line 402-409: The current seeding writes
nodePoolAnnotationCurrentRolloutConfig using token.RolloutHashWithoutVersion()
unconditionally, which can suppress user-driven rollouts; change the logic in
the reconcile path that sets nodePoolAnnotationCurrentRolloutConfig so it
prefers the last-applied/legacy value (check any legacy annotation like the
prior "lastApplied" key or bootstrap state) and only auto-seed from
token.RolloutHashWithoutVersion() when you can prove no rollout is pending
(e.g., verify isUpdatingConfig() is false or compare current vs target rollout
hashes and ensure they differ/are stable). Update the code that currently
checks/creates nodePool.Annotations and assigns
nodePoolAnnotationCurrentRolloutConfig to first try restoring from
legacy/last-applied state, and fallback to auto-seed from
token.RolloutHashWithoutVersion() only when no pending rollout is detected.

In `@test/e2e/v2/tests/nodepool_rollout_control_test.go`:
- Around line 67-174: The test ManagementImageChangeNoRolloutTest is mutating
the shared default NodePool via getDefaultNodePool; instead create an isolated
NodePool using buildTestNodePool (mirroring SpecDrivenChangeTriggersRolloutTest)
and register cleanup with DeferCleanup to call cleanupNodePool(...) so the test
operates on its own NodePool; update the teardown logic to only remove the
HAProxy annotation on the test NodePool (no IsNotFound branch) and use the test
NodePool object in all subsequent references instead of
defaultNP/getDefaultNodePool.
- Around line 287-396: The test OperatorUpgradeNoRolloutTest mutates the shared
default NodePool via getDefaultNodePool; change it to create and use a dedicated
test NodePool like SpecDrivenChangeTriggersRolloutTest does by calling
buildTestNodePool (or the same factory used there), wait for the NodePool to
become ready (so annotations are seeded), then remove the rollout annotation on
that test NodePool to simulate pre-upgrade state, and register cleanup with
DeferCleanup to call cleanupNodePool (or the same cleanup helper) to delete the
test NodePool and restore state; ensure all references to np and baseline node
lists use the newly created NodePool instead of getDefaultNodePool.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: ad3269e8-032f-403d-8aef-f75206c60cb9

📥 Commits

Reviewing files that changed from the base of the PR and between 8ea786c and 8d9fd1a.

📒 Files selected for processing (7)

• hypershift-operator/controllers/nodepool/capi.go
• hypershift-operator/controllers/nodepool/conditions.go
• hypershift-operator/controllers/nodepool/config.go
• hypershift-operator/controllers/nodepool/config_test.go
• hypershift-operator/controllers/nodepool/nodepool_controller.go
• test/e2e/v2/tests/nodepool_lifecycle_test.go
• test/e2e/v2/tests/nodepool_rollout_control_test.go

[codecov]

Codecov Report

❌ Patch coverage is 84.52381% with 39 lines in your changes missing coverage. Please review.
✅ Project coverage is 47.16%. Comparing base (286fb69) to head (c010b61).
⚠️ Report is 62 commits behind head on main.

Files with missing lines	Patch %	Lines
hypershift-operator/controllers/nodepool/capi.go	92.24%	9 Missing and 1 partial ⚠️
hypershift-operator/controllers/nodepool/token.go	52.38%	8 Missing and 2 partials ⚠️
hypershift-operator/controllers/nodepool/config.go	83.33%	5 Missing and 2 partials ⚠️
...erator/controllers/nodepool/nodepool_controller.go	46.15%	7 Missing ⚠️
...rshift-operator/controllers/nodepool/conditions.go	90.62%	2 Missing and 1 partial ⚠️
.../karpenterignition/karpenterignition_controller.go	86.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8698      +/-   ##
==========================================
+ Coverage   47.11%   47.16%   +0.05%     
==========================================
  Files         786      786              
  Lines       99220    99334     +114     
==========================================
+ Hits        46744    46850     +106     
- Misses      49317    49320       +3     
- Partials     3159     3164       +5     

Files with missing lines	Coverage Δ
.../karpenterignition/karpenterignition_controller.go	65.03% <86.66%> (+0.34%)	⬆️
...rshift-operator/controllers/nodepool/conditions.go	60.87% <90.62%> (+0.98%)	⬆️
hypershift-operator/controllers/nodepool/config.go	81.45% <83.33%> (-0.82%)	⬇️
...erator/controllers/nodepool/nodepool_controller.go	43.76% <46.15%> (-0.03%)	⬇️
hypershift-operator/controllers/nodepool/capi.go	74.80% <92.24%> (+2.01%)	⬆️
hypershift-operator/controllers/nodepool/token.go	80.58% <52.38%> (-1.46%)	⬇️

... and 1 file with indirect coverage changes

Flag	Coverage Δ
cmd-support	40.84% <ø> (ø)
cpo-hostedcontrolplane	50.33% <ø> (+<0.01%)	⬆️
cpo-other	47.60% <ø> (ø)
hypershift-operator	57.38% <84.38%> (+0.14%)	⬆️
other	34.73% <86.66%> (+0.03%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[openshift-ci]

Stale PRs are closed after 21d of inactivity.

If this PR is still relevant, comment to refresh it or remove the stale label.
Mark the PR as fresh by commenting /remove-lifecycle stale.

If this PR is safe to close now please do so with /close.

/lifecycle stale

[hypershift-jira-solve-ci]

This confirms the root cause. The PR added new tests in config_test.go that correctly use nodePoolAnnotationCurrentRolloutConfig, but failed to update two existing test files (nodepool_controller_test.go and conditions_test.go) which still use the old nodePoolAnnotationCurrentConfig annotation.

Test Failure Analysis Complete

Job Information

• Prow Job: Unit Tests (hypershift-operator)
• Build ID: 27175716518 / Job 80224268250
• Target: hypershift-operator/controllers/nodepool
• PR: CNTRLPLANE-3632: Predictable NodePool rollout control #8698 — OCPSTRAT-3298: Predictable NodePool rollout control
• Commit: 8d9fd1a

Test Failure Analysis

Error

--- FAIL: TestIsUpdatingConfig/it_is_updating_when_strings_does_not_match (0.00s)
    nodepool_controller_test.go:87:
        Expected
            <bool>: false
        to equal
            <bool>: true

--- FAIL: TestUpdatingConfigCondition/NodePool_is_Replace_and_updating_config (0.11s)
    conditions_test.go:248:
        Expected
            <v1.ConditionStatus>: False
        to equal
            <v1.ConditionStatus>: True

Summary

Two unit tests fail because the PR changed isUpdatingConfig() to read from a new annotation (nodePoolAnnotationCurrentRolloutConfig) but did not update the existing tests in nodepool_controller_test.go and conditions_test.go, which still populate the old annotation (nodePoolAnnotationCurrentConfig). Since the new annotation is empty in these tests, the new early-return guard (if currentHash == "" { return false }) fires, causing isUpdatingConfig to always return false — breaking both assertions that expect true.

Root Cause

The PR introduces a new "rollout config" concept to separate management-side config changes (e.g., HAProxy image bumps) from spec-driven config changes that require node replacement. This involved:

1. New annotation: nodePoolAnnotationCurrentRolloutConfig (hypershift.openshift.io/nodePoolCurrentRolloutConfig) replaces nodePoolAnnotationCurrentConfig for rollout decisions.

2. Changed isUpdatingConfig() function (in nodepool_controller.go): Now reads from nodePoolAnnotationCurrentRolloutConfig instead of nodePoolAnnotationCurrentConfig, and adds an early-return false when the new annotation is empty (to prevent spurious rollouts on first reconcile).

3. Changed updatingConfigCondition() function (in conditions.go): Now calls token.RolloutHashWithoutVersion() and reads from nodePoolAnnotationCurrentRolloutConfig.

The two tests were not updated:

• TestIsUpdatingConfig (nodepool_controller_test.go:87): The "it is updating when strings does not match" test case sets nodePoolAnnotationCurrentConfig: "config1" on the NodePool, then calls isUpdatingConfig(nodePool, "config2") expecting true. But isUpdatingConfig() now reads nodePoolAnnotationCurrentRolloutConfig, which is empty → early-return false.

• TestUpdatingConfigCondition (conditions_test.go:248): The "NodePool is Replace and updating config" test case sets nodePoolAnnotationCurrentConfig: "08e4f890" in the NodePool annotations. When updatingConfigCondition() runs, it calls isUpdatingConfig() which reads the empty nodePoolAnnotationCurrentRolloutConfig → returns false → condition status is set to False instead of the expected True.

The PR added correct new tests in config_test.go that use nodePoolAnnotationCurrentRolloutConfig, but the two pre-existing test files were missed.

Recommendations

1. Fix TestIsUpdatingConfig in nodepool_controller_test.go: Change the test case annotation from nodePoolAnnotationCurrentConfig to nodePoolAnnotationCurrentRolloutConfig:

   // Before:
   Annotations: map[string]string{
       nodePoolAnnotationCurrentConfig: "config1",
   },
   // After:
   Annotations: map[string]string{
       nodePoolAnnotationCurrentRolloutConfig: "config1",
   },

   Also update the "not updating" test case to use the new annotation for consistency.

2. Fix TestUpdatingConfigCondition in conditions_test.go: Change line 192 from:

   nodePoolAnnotationCurrentConfig: "08e4f890",

   to:

   nodePoolAnnotationCurrentRolloutConfig: "08e4f890",

   This ensures the isUpdatingConfig() function finds a non-empty value and performs the comparison.

3. Consider also testing the empty-annotation guard: Add an explicit test case in TestIsUpdatingConfig that verifies isUpdatingConfig returns false when nodePoolAnnotationCurrentRolloutConfig is absent — this documents the intentional "first reconcile" behavior.

Evidence

Evidence	Detail
Failed test 1	TestIsUpdatingConfig/it_is_updating_when_strings_does_not_match at nodepool_controller_test.go:87 — expected true, got false
Failed test 2	TestUpdatingConfigCondition/NodePool_is_Replace_and_updating_config at conditions_test.go:248 — expected condition status True, got False
Root cause code change	isUpdatingConfig() in nodepool_controller.go:740 now reads nodePoolAnnotationCurrentRolloutConfig instead of nodePoolAnnotationCurrentConfig
Guard clause	isUpdatingConfig() returns false when nodePoolAnnotationCurrentRolloutConfig is empty (new early-return at line 741-743)
Test annotation (test 1)	nodepool_controller_test.go:74 sets nodePoolAnnotationCurrentConfig: "config1" — old annotation, not read by new code
Test annotation (test 2)	conditions_test.go:192 sets nodePoolAnnotationCurrentConfig: "08e4f890" — old annotation, not read by new code
PR correctly added new tests	config_test.go has new tests using nodePoolAnnotationCurrentRolloutConfig (lines 623+), confirming the new annotation is correct
Package	github.com/openshift/hypershift/hypershift-operator/controllers/nodepool — 7.901s, FAIL

---

[openshift-ci]

Stale PRs rot after 14d of inactivity.

Mark the PR as fresh by commenting /remove-lifecycle rotten.
Rotten PRs close after an additional 7d of inactivity.

If this PR is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

[csrwng]

/remove-lifecycle rotten

[openshift-ci-robot]

@csrwng: This pull request references CNTRLPLANE-3632 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Decouples NodePool rollout triggering from management-side configuration changes (e.g. HAProxy image digest bumps) so that only user-driven spec changes cause worker node replacement.

Today the NodePool controller uses a single hash over the entire rendered ignition config — including management-side image references — to drive rollout decisions. Any change to this hash triggers a full Replace/InPlace rollout. This means automated HAProxy image updates (which happen on every HyperShift operator upgrade) cause unnecessary node churn.

This PR introduces:

• RolloutHash() / RolloutHashWithoutVersion(): New hash methods that include only spec-driven inputs (user MachineConfigs, release version, pull secret, trust bundle, global config), excluding management-side content like HAProxy
• nodePoolCurrentRolloutConfig annotation: Tracks the current rollout config hash, used for rollout decisions instead of comparing data secret names
• Annotation seeding: On first reconcile after operator upgrade, the annotation is populated without triggering a rollout
• isUpdatingConfig() safety: Returns false when the annotation is absent to prevent condition flip-flop during upgrade

The existing Hash() and payload secret naming are unchanged — new nodes always get the latest payload including management-side content when they ARE replaced for spec-driven reasons.

Which issue(s) this PR fixes:

Fixes OCPSTRAT-3298

Special notes for your reviewer:

• RolloutHashWithoutVersion() intentionally includes globalConfig unlike the existing HashWithoutVersion() which omits it. Proxy/image config changes should trigger rollouts.
• The parse() → doParse() refactor is necessary because haproxy content is prepended inside parse() and cannot be stripped from the final string after the fact.
• The secret_janitor_test.go build error is pre-existing and unrelated to this PR.

Checklist:

• Subject and description added to both, commit and PR.
• Relevant issues have been referenced.
• This change includes docs.
• This change includes unit tests.

Summary by CodeRabbit

Release Notes

• New Features

• Added rollout configuration annotation to track and manage NodePool configuration state across updates.

• Bug Fixes

• Management-side HAProxy or image digest changes no longer unnecessarily trigger node rollovers.

• Configuration updates now spec-driven for more predictable rollout behavior.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: csrwng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [csrwng]
• api/OWNERS [csrwng]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@api/hypershift/v1beta1/nodepool_conditions.go`:
- Around line 91-95: The comment for NodePoolConfigUpdatePendingConditionType
inaccurately describes scale-up behavior. Update it to state that new or
replaced nodes may receive the latest management-side configuration while
existing nodes retain the previous configuration until the next spec-driven
rollout, without changing payload hashing or secret naming.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: ba91d631-99b7-469c-a822-dfb15d50706c

📥 Commits

Reviewing files that changed from the base of the PR and between 8d9fd1a and b2abede.

📒 Files selected for processing (1)

• api/hypershift/v1beta1/nodepool_conditions.go

[csrwng]

/rebase

[github-actions]

🤖 Rebasing PR onto main: workflow run

[csrwng]

/rebase

[github-actions]

🤖 Rebasing PR onto main: workflow run

[csrwng]

/test unit

[muraee]

/lgtm

[openshift-merge-bot]

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-5-0
/test e2e-aws-5-0
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-azure-self-managed
/test e2e-v2-gke

[csrwng]

Retrying flakes:

hypershift-e2e: [It] [sig-hypershift][Jira:Hypershift] Karpenter [Feature:AutoNode] Karpenter Upgrade should upgrade the control plane and drift Karpenter nodes to the new version [lifecycle, karpenter-upgrade, Informing] expand_less 1h15m45s
{Failed to wait for 1 nodes to become ready in 45m0s: context deadline exceeded failed [FAILED] Failed to wait for 1 nodes to become ready in 45m0s: context deadline exceeded
In [It] at: /hypershift/test/e2e/util/util.go:612 @ 09/09/26 12:24:43.461
}

/test e2e-v2-aws

TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster/EnsureNoCrashingPods expand_less 0s
{Failed === RUN TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster/EnsureNoCrashingPods
util.go:858: Container snapshot-controller in pod csi-snapshot-controller-6d8f7bbc4-k4cbt has a restartCount > 0 (1)
--- FAIL: TestCreateClusterHABreakGlassCredentials/ValidateHostedCluster/EnsureNoCrashingPods (0.09s)
}

/test e2e-aws

nodepool_rolling_upgrade_test.go:125: AllMachinesReady=False: WaitingForInfrastructure(1 of 3 machines are not ready

    Machine node-pool-zvpkv-test-rolling-upgrade-pnsnw-snx55: WaitingForInfrastructure: 
    )
nodepool_rolling_upgrade_test.go:125: AllNodesHealthy=False: WaitingForNodeRef(1 of 3 machines are not healthy
    Machine node-pool-zvpkv-test-rolling-upgrade-pnsnw-snx55: WaitingForNodeRef
    )

/test e2e-aks-5-0

[openshift-ci]

@csrwng: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/rosa-e2e-images	c010b61	link	true	/test rosa-e2e-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci]

PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[csrwng]

/rebase

[github-actions]

🤖 Rebasing PR onto main: workflow run