Skip to content

CNTRLPLANE-3599: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump - #8683

Merged
openshift-merge-bot[bot] merged 4 commits into
openshift:mainfrom
bryan-cox:k8s-bump
Jun 5, 2026
Merged

CNTRLPLANE-3599: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump#8683
openshift-merge-bot[bot] merged 4 commits into
openshift:mainfrom
bryan-cox:k8s-bump

Conversation

@bryan-cox

@bryan-cox bryan-cox commented Jun 5, 2026

Copy link
Copy Markdown
Member

What this PR does / why we need it:

Removes the replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.19.7 directive from go.mod that was blocking the k8s 1.36 dependency bump. The pin was originally added because webhook.Validator deprecation in v0.20 broke vendored dependencies, but all CAPI providers and HyperShift code have since migrated to CustomValidator/CustomDefaulter.

Changes:

  • Remove the controller-runtime v0.19.7 replace directive, upgrading to v0.22.4
  • Bump cluster-node-tuning-operator to release-4.20 (uses new webhook.CustomValidator interface)
  • Bump k8s.io/kubelet from v0.32.2 to v0.33.3
  • Fix controller.NewUnmanaged() call in proxy controller (v0.22 dropped the manager.Manager parameter)
  • Adapt test assertions for controller-runtime v0.22 fake client behavior changes:
    • ManagedFields validation requires APIVersion and FieldsType
    • TypeMeta cleared on Get (matching real API server)
    • Empty list results return nil Items instead of empty slice

Which issue(s) this PR fixes:

Unblocks the k8s 1.36 dependency rebase for HyperShift.

Special notes for your reviewer:

The original concern noted in the replace directive comment — "webhook.Validator deprecation in v0.20 breaks everything, conversion is nontrivial" — turned out to be already resolved. Only 2 compile errors and 4 test adaptation issues remained, all straightforward fixes.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests

    • Improved test robustness: normalized empty list results, aligned managed-fields fixtures, relaxed TypeMeta expectations, and updated requeue assertions to validate requeue-after durations.
  • Bug Fixes

    • Retry behavior refined for conflict cases: controllers now requeue with a short delay to reduce transient failures.
  • Chores

    • Updated project dependencies to newer versions for compatibility and stability.

bryan-cox added 3 commits June 5, 2026 07:37
Remove the replace directive that pinned controller-runtime to v0.19.7.
The pin was added when webhook.Validator deprecation in v0.20 broke
vendored dependencies, but all CAPI providers and HyperShift code have
since migrated to CustomValidator/CustomDefaulter.

Bump cluster-node-tuning-operator to release-4.20 branch which uses
the new webhook.CustomValidator interface compatible with v0.22.
This also pulls in k8s.io/kubelet v0.33.3 (from v0.32.2).

Signed-off-by: Bryan Cox <brcox@redhat.com>
Commit-Message-Assisted-by: Claude (via Claude Code)
Revendor after removing the controller-runtime v0.19.7 replace
directive. Updates:
- sigs.k8s.io/controller-runtime v0.19.7 -> v0.22.4
- cluster-node-tuning-operator to release-4.20 branch
- k8s.io/kubelet v0.32.2 -> v0.33.3
- golang.org/x/exp/maps added as new NTO dependency

Regenerated test fixtures with updated resourceVersion values.

Signed-off-by: Bryan Cox <brcox@redhat.com>
Commit-Message-Assisted-by: Claude (via Claude Code)
- Remove mgr argument from controller.NewUnmanaged() call in proxy
  controller (v0.22 dropped the manager.Manager parameter)
- Add APIVersion and FieldsType to ManagedFieldsEntry in apply_test
  (v0.22 fake client validates these fields)
- Remove TypeMeta from expected objects in annotation enforcer test
  (v0.22 fake client clears TypeMeta on Get, matching real API server)

Signed-off-by: Bryan Cox <brcox@redhat.com>
Commit-Message-Assisted-by: Claude (via Claude Code)
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 5, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This pull request explicitly references no jira issue.

Details

In response to this:

What this PR does / why we need it:

Removes the replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.19.7 directive from go.mod that was blocking the k8s 1.36 dependency bump. The pin was originally added because webhook.Validator deprecation in v0.20 broke vendored dependencies, but all CAPI providers and HyperShift code have since migrated to CustomValidator/CustomDefaulter.

Changes:

  • Remove the controller-runtime v0.19.7 replace directive, upgrading to v0.22.4
  • Bump cluster-node-tuning-operator to release-4.20 (uses new webhook.CustomValidator interface)
  • Bump k8s.io/kubelet from v0.32.2 to v0.33.3
  • Fix controller.NewUnmanaged() call in proxy controller (v0.22 dropped the manager.Manager parameter)
  • Adapt test assertions for controller-runtime v0.22 fake client behavior changes:
  • ManagedFields validation requires APIVersion and FieldsType
  • TypeMeta cleared on Get (matching real API server)
  • Empty list results return nil Items instead of empty slice

Which issue(s) this PR fixes:

Unblocks the k8s 1.36 dependency rebase for HyperShift.

Special notes for your reviewer:

The original concern noted in the replace directive comment — "webhook.Validator deprecation in v0.20 breaks everything, conversion is nontrivial" — turned out to be already resolved. Only 2 compile errors and 4 test adaptation issues remained, all straightforward fixes.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. do-not-merge/needs-area labels Jun 5, 2026
@coderabbitai

coderabbitai Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR updates Go module requirements (bumping OpenShift modules and k8s.io/kubelet; removing a controller-runtime replace), changes AzurePrivateLinkService controller conflict handling to requeue after one second (and updates tests), normalizes nil vs empty slices in several infra controller tests before YAML diffs, adjusts a managed-fields test fixture, and switches a proxy controller instantiation to use controller.NewUnmanaged.

Sequence Diagram(s)

sequenceDiagram
  participant AzurePLSController
  participant KubernetesAPI
  participant ReconcilerResult
  AzurePLSController->>KubernetesAPI: patch finalizer (add/remove)
  KubernetesAPI-->>AzurePLSController: conflict error
  AzurePLSController->>ReconcilerResult: return RequeueAfter = 1s
Loading

Suggested reviewers

  • sjenning
  • clebs
🚥 Pre-merge checks | ✅ 11
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies only standard Go tests using testing package, not Ginkgo framework. No Ginkgo test declarations (It, Describe, Context, When) found. Check is not applicable.
Test Structure And Quality ✅ Passed All tests follow quality standards: table-driven single-responsibility patterns, proper fake client setup, meaningful assertion messages, and consistent repository conventions.
Topology-Aware Scheduling Compatibility ✅ Passed PR includes only dependency updates, controller-runtime API signature fixes, and test adaptations. No topology-unaware scheduling constraints introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed No new Ginkgo e2e tests are added in this PR. All changes are unit tests and controller code in non-e2e directories. The check only applies when new e2e tests are added.
No-Weak-Crypto ✅ Passed No weak cryptographic algorithms (MD5, SHA1, DES, RC4, 3DES, Blowfish, ECB), custom crypto implementations, or non-constant-time secret comparisons detected in any modified files.
Container-Privileges ✅ Passed PR modifies only Go code and go.mod (dependencies), not container manifests. No privilege settings (privileged, hostPID, hostNetwork, hostIPC, SYS_ADMIN, allowPrivilegeEscalation) introduced.
No-Sensitive-Data-In-Logs ✅ Passed PR contains only test updates, dependency bumps, and requeue timing changes. No logging of sensitive data (passwords, tokens, API keys, PII) added or exposed.
Title check ✅ Passed The title accurately reflects the main change: removing the controller-runtime v0.19.7 pin to enable Kubernetes 1.36 compatibility. It is concise, specific, and clearly summarizes the primary objective of the PR.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci

openshift-ci Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bryan-cox

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release and removed do-not-merge/needs-area labels Jun 5, 2026
@bryan-cox
bryan-cox marked this pull request as ready for review June 5, 2026 11:48
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 5, 2026
@openshift-ci
openshift-ci Bot requested review from sdminonne and sjenning June 5, 2026 11:49
@bryan-cox bryan-cox changed the title NO-JIRA: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump OCPSTRAT-3036: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump Jun 5, 2026
@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 41.43%. Comparing base (f13c62d) to head (0132ec7).

Files with missing lines Patch % Lines
...ypershift-operator/controllers/proxy/reconciler.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #8683   +/-   ##
=======================================
  Coverage   41.43%   41.43%           
=======================================
  Files         756      756           
  Lines       93647    93647           
=======================================
  Hits        38802    38802           
  Misses      52124    52124           
  Partials     2721     2721           
Files with missing lines Coverage Δ
.../controllers/azureprivatelinkservice/controller.go 92.12% <100.00%> (ø)
...ypershift-operator/controllers/proxy/reconciler.go 0.00% <0.00%> (ø)
Flag Coverage Δ
cmd-support 34.87% <ø> (ø)
cpo-hostedcontrolplane 43.50% <ø> (ø)
cpo-other 42.74% <100.00%> (ø)
hypershift-operator 51.57% <0.00%> (ø)
other 31.64% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@openshift-ci openshift-ci Bot added the area/platform/azure PR/issue for Azure (AzurePlatform) platform label Jun 5, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 5, 2026

Copy link
Copy Markdown

@bryan-cox: This pull request references OCPSTRAT-3036 which is a valid jira issue.

Details

In response to this:

What this PR does / why we need it:

Removes the replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.19.7 directive from go.mod that was blocking the k8s 1.36 dependency bump. The pin was originally added because webhook.Validator deprecation in v0.20 broke vendored dependencies, but all CAPI providers and HyperShift code have since migrated to CustomValidator/CustomDefaulter.

Changes:

  • Remove the controller-runtime v0.19.7 replace directive, upgrading to v0.22.4
  • Bump cluster-node-tuning-operator to release-4.20 (uses new webhook.CustomValidator interface)
  • Bump k8s.io/kubelet from v0.32.2 to v0.33.3
  • Fix controller.NewUnmanaged() call in proxy controller (v0.22 dropped the manager.Manager parameter)
  • Adapt test assertions for controller-runtime v0.22 fake client behavior changes:
  • ManagedFields validation requires APIVersion and FieldsType
  • TypeMeta cleared on Get (matching real API server)
  • Empty list results return nil Items instead of empty slice

Which issue(s) this PR fixes:

Unblocks the k8s 1.36 dependency rebase for HyperShift.

Special notes for your reviewer:

The original concern noted in the replace directive comment — "webhook.Validator deprecation in v0.20 breaks everything, conversion is nontrivial" — turned out to be already resolved. Only 2 compile errors and 4 test adaptation issues remained, all straightforward fixes.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests

  • Improved unit test robustness: normalize empty list results, align managed-fields fixtures, and relax TypeMeta expectations.

  • Bug Fixes

  • Retry behavior improved for certain controller conflict cases: operations now requeue after a short delay to reduce transient failures.

  • Chores

  • Updated project dependencies to newer versions for compatibility and stability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@control-plane-operator/controllers/azureprivatelinkservice/controller.go`:
- Line 272: Add a short code comment above each conflict-handling return that
currently uses ctrl.Result{RequeueAfter: time.Second} (the branches checking
apierrors.IsConflict(err) during finalizer updates in azureprivatelinkservice
controller.go) that explains why a fixed 1s delay is required instead of
ctrl.Result{Requeue: true} (e.g., to allow in-flight finalizer updates or
related cache propagation to settle and avoid hot-looping), and state the
intended behavior and any assumptions (retry interval tolerances, expected
conflict frequency); also update the PR/commit message to include the same
rationale so reviewers can see the justification.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: e662d8f4-ffaa-4b7b-adf8-f656b39b73be

📥 Commits

Reviewing files that changed from the base of the PR and between 8ce2079 and b8da255.

📒 Files selected for processing (3)
  • control-plane-operator/controllers/azureprivatelinkservice/controller.go
  • control-plane-operator/controllers/azureprivatelinkservice/controller_test.go
  • control-plane-operator/controllers/hostedcontrolplane/infra/infra_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • control-plane-operator/controllers/hostedcontrolplane/infra/infra_test.go

- Normalize nil Items slices to empty slices in infra tests
  (v0.22 fake client returns nil for empty list results)
- Replace deprecated Requeue with RequeueAfter in Azure PLS controller
  conflict handling (Requeue field removed in controller-runtime v0.22)

Signed-off-by: Bryan Cox <brcox@redhat.com>
Commit-Message-Assisted-by: Claude (via Claude Code)
@openshift-ci openshift-ci Bot added the area/platform/gcp PR/issue for GCP (GCPPlatform) platform label Jun 5, 2026
@bryan-cox bryan-cox changed the title OCPSTRAT-3036: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump CNTRLPLANE-3598: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump Jun 5, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 5, 2026

Copy link
Copy Markdown

@bryan-cox: This pull request references CNTRLPLANE-3598 which is a valid jira issue.

Details

In response to this:

What this PR does / why we need it:

Removes the replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.19.7 directive from go.mod that was blocking the k8s 1.36 dependency bump. The pin was originally added because webhook.Validator deprecation in v0.20 broke vendored dependencies, but all CAPI providers and HyperShift code have since migrated to CustomValidator/CustomDefaulter.

Changes:

  • Remove the controller-runtime v0.19.7 replace directive, upgrading to v0.22.4
  • Bump cluster-node-tuning-operator to release-4.20 (uses new webhook.CustomValidator interface)
  • Bump k8s.io/kubelet from v0.32.2 to v0.33.3
  • Fix controller.NewUnmanaged() call in proxy controller (v0.22 dropped the manager.Manager parameter)
  • Adapt test assertions for controller-runtime v0.22 fake client behavior changes:
  • ManagedFields validation requires APIVersion and FieldsType
  • TypeMeta cleared on Get (matching real API server)
  • Empty list results return nil Items instead of empty slice

Which issue(s) this PR fixes:

Unblocks the k8s 1.36 dependency rebase for HyperShift.

Special notes for your reviewer:

The original concern noted in the replace directive comment — "webhook.Validator deprecation in v0.20 breaks everything, conversion is nontrivial" — turned out to be already resolved. Only 2 compile errors and 4 test adaptation issues remained, all straightforward fixes.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests

  • Improved test robustness: normalized empty list results, aligned managed-fields fixtures, relaxed TypeMeta expectations, and updated requeue assertions to validate requeue-after durations.

  • Bug Fixes

  • Retry behavior refined for conflict cases: controllers now requeue with a short delay to reduce transient failures.

  • Chores

  • Updated project dependencies to newer versions for compatibility and stability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bryan-cox bryan-cox changed the title CNTRLPLANE-3598: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump CNTRLPLANE-3599: Remove controller-runtime v0.19.7 pin for k8s 1.36 bump Jun 5, 2026
@openshift-ci-robot

openshift-ci-robot commented Jun 5, 2026

Copy link
Copy Markdown

@bryan-cox: This pull request references CNTRLPLANE-3599 which is a valid jira issue.

Details

In response to this:

What this PR does / why we need it:

Removes the replace sigs.k8s.io/controller-runtime => sigs.k8s.io/controller-runtime v0.19.7 directive from go.mod that was blocking the k8s 1.36 dependency bump. The pin was originally added because webhook.Validator deprecation in v0.20 broke vendored dependencies, but all CAPI providers and HyperShift code have since migrated to CustomValidator/CustomDefaulter.

Changes:

  • Remove the controller-runtime v0.19.7 replace directive, upgrading to v0.22.4
  • Bump cluster-node-tuning-operator to release-4.20 (uses new webhook.CustomValidator interface)
  • Bump k8s.io/kubelet from v0.32.2 to v0.33.3
  • Fix controller.NewUnmanaged() call in proxy controller (v0.22 dropped the manager.Manager parameter)
  • Adapt test assertions for controller-runtime v0.22 fake client behavior changes:
  • ManagedFields validation requires APIVersion and FieldsType
  • TypeMeta cleared on Get (matching real API server)
  • Empty list results return nil Items instead of empty slice

Which issue(s) this PR fixes:

Unblocks the k8s 1.36 dependency rebase for HyperShift.

Special notes for your reviewer:

The original concern noted in the replace directive comment — "webhook.Validator deprecation in v0.20 breaks everything, conversion is nontrivial" — turned out to be already resolved. Only 2 compile errors and 4 test adaptation issues remained, all straightforward fixes.

Checklist:

  • Subject and description added to both, commit and PR.
  • Relevant issues have been referenced.
  • This change includes docs.
  • This change includes unit tests.

Summary by CodeRabbit

  • Tests

  • Improved test robustness: normalized empty list results, aligned managed-fields fixtures, relaxed TypeMeta expectations, and updated requeue assertions to validate requeue-after durations.

  • Bug Fixes

  • Retry behavior refined for conflict cases: controllers now requeue with a short delay to reduce transient failures.

  • Chores

  • Updated project dependencies to newer versions for compatibility and stability.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@bryan-cox

Copy link
Copy Markdown
Member Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@bryan-cox

Copy link
Copy Markdown
Member Author

/verified by e2e

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jun 5, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@bryan-cox: This PR has been marked as verified by e2e.

Details

In response to this:

/verified by e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2062887461545054208 | Cost: $3.1537217500000008 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

if err := r.Update(ctx, azPLS); err != nil {
if apierrors.IsConflict(err) {
return ctrl.Result{Requeue: true}, nil
return ctrl.Result{RequeueAfter: time.Second}, nil

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requeue is deprecated in controller-runtime v0.22.4 — the // Deprecated: Use RequeueAfter instead. annotation at vendor/sigs.k8s.io/controller-runtime/pkg/reconcile/reconcile.go:41 triggers staticcheck SA1019, which make lint enforces.

Upstream source: https://github.com/kubernetes-sigs/controller-runtime/blob/v0.22.4/pkg/reconcile/reconcile.go#L31-L42


AI-assisted response via Claude Code

name: capi-provider
namespace: hcp-namespace
resourceVersion: "1"
resourceVersion: "2"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why this change?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are auto-generated golden test fixtures managed by support/testutil/testutil.go:27 (CompareWithFixture), regenerated via UPDATE=true go test.

The test at hypershift-operator/controllers/hostedcluster/hostedcluster_controller_test.go:5289 reconciles each component twice (for range 2). The fake client sets rv="1" on Create (vendor/.../fake/client.go:421) and increments on Update (vendor/.../fake/client.go:600). With controller-runtime v0.22.4, the second reconcile pass now triggers an Update, bumping rv from "1" to "2". All 4 component fixtures changed identically.


AI-assisted response via Claude Code

@cwbotbot

cwbotbot commented Jun 5, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

e2e-aks

@hypershift-jira-solve-ci

hypershift-jira-solve-ci Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Now I have all the evidence needed to produce the final report.

Test Failure Analysis Complete

Job Information

Test Failure Analysis

Error

Failed to wait for kubeconfig to be published for HostedCluster e2e-clusters-6hn7j/request-serving-isolation-5g7bc in 10m0s: context deadline exceeded

Conditions at timeout:
- Degraded=True: UnavailableReplicas([capi-provider deployment has 1 unavailable replicas, kube-apiserver deployment has 2 unavailable replicas])
- KubeAPIServerAvailable=False: WaitingForAvailable(Waiting for Kube APIServer deployment to become available)
- AWSEndpointAvailable=False: AWSError(cannot list security groups: ...unable to read file at /var/run/secrets/openshift/serviceaccount/token: no such file or directory)
- Available=False: KubeconfigWaitingForCreate(Waiting for hosted control plane kubeconfig to be created)

Summary

This is a pre-existing flaky test unrelated to the PR changes. The TestCreateClusterRequestServingIsolation test creates a hosted cluster with request-serving node topology on a management cluster. This cluster was the last (16th of 16) hosted cluster created in a highly parallel test run (20 concurrent tests). The hosted control plane was making progress — between the first observation (RV 64479) and the second (RV 90949), the KubeAPIServer went from NotFound to WaitingForAvailable — but it didn't become fully available within the 10-minute timeout. The capi-provider pod failed to become ready (with a projected service account token not yet mounted), blocking the full control plane rollout. All 613 other tests passed, including 6 other TestCreateCluster* variants that all created hosted clusters successfully.

Root Cause

The failure is a timing/resource-pressure flake in TestCreateClusterRequestServingIsolation, not a regression from this PR's controller-runtime v0.19.7→v0.22.4 upgrade.

Why the test failed:

  1. The request-serving-isolation test first creates 5 management cluster node pools (2 request-serving + 3 non-request-serving), waits for all nodes to be ready, then creates the hosted cluster. This setup phase means the hosted cluster creation starts late relative to other tests.
  2. The hosted cluster (request-serving-isolation-5g7bc) was the 16th and last cluster created, at a time when 15 other hosted control planes were already running or being torn down on the management cluster.
  3. Under this resource pressure, the capi-provider deployment had 1 unavailable replica — the pod's projected service account token (/var/run/secrets/openshift/serviceaccount/token) was not yet mounted, causing AWS credential refresh failures.
  4. Without a functioning capi-provider, the kube-apiserver deployment couldn't complete its rollout. The cluster was progressing (KubeAPIServer status went from NotFoundWaitingForAvailable) but too slowly to meet the 10-minute timeout.

Why this is NOT caused by the PR:

  • The PR changes Requeue: trueRequeueAfter: time.Second in Azure PLS controller (not involved in this AWS test)
  • The controller.NewUnmanaged() signature change in the proxy controller is a no-op behavioral change
  • Test fixture resourceVersion bumps ("1"→"2") reflect fake client behavior changes, not functional changes
  • ManagedFields test adaptations add required fields for v0.22 validation
  • None of the changed code paths touch capi-provider, kube-apiserver scheduling, request-serving topology, or AWS endpoint handling
  • All 6 other TestCreateCluster* tests passed, showing the controller-runtime upgrade doesn't affect hosted cluster creation
Recommendations
  1. Retest the PR — this failure is a known flake pattern in the request-serving-isolation test, not caused by the PR changes. A /retest should pass.
  2. Consider filing a flake issue for TestCreateClusterRequestServingIsolation — the 10-minute timeout for kubeconfig publication may be too tight when the management cluster is under heavy load from 15+ concurrent hosted control planes. The test consistently creates its hosted cluster last due to the node pool setup phase.
  3. No code changes needed — the PR's controller-runtime v0.19.7→v0.22.4 migration is correct and unrelated to this failure.
Evidence
Evidence Detail
Test result 615 tests, 30 skipped, 2 failures (both from TestCreateClusterRequestServingIsolation)
Cluster creation order request-serving-isolation-5g7bc was the 16th (last) of 16 hosted clusters created
Progress between checks KubeAPIServer went from NotFound (RV 64479) → WaitingForAvailable (RV 90949) showing the cluster was progressing
Timeout 10m0s waiting for kubeconfig to be published
capi-provider status 1 unavailable replica — projected SA token not yet mounted at /var/run/secrets/openshift/serviceaccount/token
Other CreateCluster tests All 6 other TestCreateCluster* variants passed (CustomConfig, Private, PrivateWithRouteKAS, HABreakGlassCredentials, CreateCluster, Proxy)
PR scope controller-runtime v0.19.7→v0.22.4 upgrade: Azure PLS requeue changes, proxy controller signature fix, test fixture updates — no code touching capi-provider, kube-apiserver, or request-serving topology
Test parallelism 20 concurrent test goroutines, creating 16 hosted clusters simultaneously

@bryan-cox

Copy link
Copy Markdown
Member Author

/retest

@csrwng

csrwng commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 5, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

@csrwng

csrwng commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-self-managed
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-azure-self-managed | Build: 2062940790698545152 | Cost: $2.022126 | Failed step: hypershift-azure-run-e2e-self-managed

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aws | Build: 2062940790673379328 | Cost: $3.4372415000000007 | Failed step: hypershift-aws-run-e2e-nested

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD f13c62d and 2 for PR HEAD 0132ec7 in total

@openshift-ci

openshift-ci Bot commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

@bryan-cox: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release area/platform/azure PR/issue for Azure (AzurePlatform) platform area/platform/gcp PR/issue for GCP (GCPPlatform) platform jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants