OCPBUGS-113993: [release-4.20] fix router component ordering to prevent missing HAProxy backends - #9399
Conversation
…-creating components Add a predicate to the router component that ensures all expected HCP router routes exist and are ready before reconciling on ARO-HCP clusters. This includes kube-apiserver-internal, konnectivity-server, oauth-internal, ignition-server, and conditionally metrics-proxy when metrics forwarding is enabled. The router also declares explicit WithDependencies on ignition-server and metrics-proxy components to give the informer cache more time to sync after those components create their routes via server-side apply. Add a TODO for introducing live reloading (like shared proxy) so the router config is updated when routes change after the initial reconcile. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> (cherry picked from commit 7820bb7)
…dencies The metrics-proxy component is conditional (only created when MetricsForwarding.Mode=Forward), so declaring it as a hard dependency causes a deadlock on platforms where it is never instantiated (e.g. GKE). The router predicate already ensures metrics-proxy routes exist before reconciling, making the explicit dependency unnecessary. Update all 5 router fixture YAML files to match the actual declared dependency (ignition-server only). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> (cherry picked from commit 0fa80da)
…AS dependency The ignition-server only needs the management cluster KAS (via its service account token), not the hosted cluster KAS. Including it in the implicit KAS dependency creates a deadlock on private clusters with KMS encryption (e.g. Azure ARO-HCP with private Key Vault): Router → ignition-server → KAS → Router (for KMS connectivity) By excluding ignition-server from the KAS dependency, its deployment and route are created independently, unblocking the router, which in turn provides the network path KAS needs to reach the KMS endpoint. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> (cherry picked from commit 25c74d3)
The oauth-internal route is only created by the infra reconciler when OAuth is enabled (HCPOAuthEnabled). When OAuth is disabled (e.g. OIDC authentication), the route is never created, causing the ARO router predicate to wait forever — a deadlock. Move oauth-internal from the unconditional base list to a conditional append gated on HCPOAuthEnabled, matching how the infra reconciler decides whether to create the route. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> (cherry picked from commit fee9dc6)
…icit KAS dependency The ignition-server-proxy component does not need the hosted KAS — it only proxies to the ignition-server, which is already excluded. Including it in the implicit KAS dependency creates the same deadlock as ignition-server on private clusters with KMS encryption: the router waits for the ignition-server-proxy Service to exist, but ignition- server-proxy waits for KAS, which waits for the router. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> (cherry picked from commit 61d8a05)
|
Pipeline controller notification For optional jobs, comment This repository is configured in: LGTM mode |
openshift-ci-robot
added
jira/valid-reference
|
@bennerv: This pull request references Jira Issue OCPBUGS-113993, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository: openshift/coderabbit/.coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
openshift-ci
Bot
added
area/control-plane-operator
- Remove endpoint_resolver and metrics_proxy references (packages do not exist on release-4.20) - Remove metricsProxyRouteRequired and related tests (MetricsForwarding API does not exist on release-4.20) - Replace azureutil.IsAroHCPByHCP with azureutil.IsAroHCP (ByHCP variant does not exist on release-4.20) - Remove AzureTopologyPrivate from test helper (type does not exist on release-4.20) - Add SwiftPodNetworkInstanceAnnotation to aroHCP test helper and SetAsAroHCPTest calls (IsPrivateHCP checks env var on release-4.20) - Use FakeReleaseProvider instead of gomock-generated mock (mock file not generated on release-4.20) - Only check ignitionserverv2.ComponentName in router ordering test (metricsproxyv2 does not exist on release-4.20) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> (cherry picked from commit cc36777)
f29c4e1 to
0574472
Compare
|
/jira refresh |
openshift-ci-robot
added
jira/valid-bug
|
@bennerv: This pull request references Jira Issue OCPBUGS-113993, which is valid. The bug has been moved to the POST state. 7 validation(s) were run on this bug
DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/retest-required |
1 similar comment
|
/retest-required |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: bennerv, bryan-cox The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
openshift-ci
Bot
added
the
approved
|
/retest-required |
1 similar comment
|
/retest-required |
|
/verified by e2e Azure/ARO-HCP#6697 |
openshift-ci-robot
added
the
verified
|
@bennerv: This PR has been marked as verified by DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
Confirmed by @twolff-gh |
|
/lgtm |
|
/retest-required |
csrwng
added
the
backport-risk-assessed
|
/retest-required |
1 similar comment
|
/retest-required |
|
/test e2e-kubevirt-aws-ovn-reduced |
|
/retest-required |
2 similar comments
|
/retest-required |
|
/retest-required |
|
@bennerv: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
6 participants
Summary
Manual cherry-pick of #9263 (release-4.21) to release-4.20. The router component could reconcile before route-creating components had created their routes, producing a router config with missing HAProxy backends and causing NodePool ignition failures on Azure due to swift NIC contention.
The automated cherry-pick could not be applied cleanly, so this PR reproduces the change manually.
Commits
fix(control-plane-operator): ensure router component runs after route-creating componentsfix(control-plane-operator): drop metrics-proxy from router WithDependenciesfix(control-plane-component): exclude ignition-server from implicit KAS dependencyfix(router): conditionally include oauth-internal in ARO route predicatefix(control-plane-component): exclude ignition-server-proxy from implicit KAS dependencyfix: adapt cherry-picked changes for release-4.20Conflict resolution / fixes applied
testdata/router/GCP/zz_fixture_TestControlPlaneComponents_router_controlplanecomponent.yaml— release-4.20 has no GCP router test case, so this fixture is not generated on this branch (modify/delete conflict).go build,go vet, the router component tests, andTestControlPlaneComponentsall pass with no fixture drift.References
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com