fix(ci): tolerate stale vcluster webhook readiness - #12278
Merged
Conversation
Signed-off-by: Lavanya <lvijayakrish@nvidia.com>
lavanyavijayk
requested review from
nv-anants,
nv-ashwinkumarr,
nv-tusharma and
sara4dev
July 28, 2026 19:56
github-actions
Bot
added
fix
documentation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (6)
WalkthroughThe Helm chart adds configurable webhook Service publication of not-ready endpoints. The setup action enables and verifies this setting, waits for controller-manager readiness, and collects expanded host and vCluster diagnostics on failure. ChangesWebhook readiness configuration
Operator setup and diagnostics
Estimated code review effort: 3 (Moderate) | ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
Contributor
|
🎯 Code Coverage (details) 🔗 Commit SHA: dc99a8f | Docs | Datadog PR Page | Give us feedback! |
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
julienmancuso
approved these changes
Jul 28, 2026
sara4dev
approved these changes
Jul 28, 2026
sara4dev
left a comment
sara4dev
left a comment
Contributor
There was a problem hiding this comment.
we are doing to unblock the CI release for now and do a proper fix later.
1 task
sara4dev
added a commit
that referenced
this pull request
Jul 29, 2026
This reverts commit 329132a. TEMPORARY -- for experimental validation on this branch only, not intended to merge as-is. #12278 works around the vCluster guest/host K8s skew rather than fixing it. Its own comment named the cause: "The vCluster syncer on AWS stalls pod-Ready propagation due to a NATS qosClass reconciliation loop, so kubectl wait against the virtual cluster never returns." That is loft-sh/vcluster#3578, which this branch fixes at the root by aligning the guest control plane to the host version. Reverting the workaround here tests whether the root-cause fix makes it unnecessary -- specifically whether the operator becomes Ready through the vCluster API without host-namespace polling, and whether the webhook Service routes without publishNotReadyAddresses=true. If the deploy tests pass without it, #12278 can be reverted on main as well. If they fail, the workaround is load-bearing for a reason beyond the version skew and this commit must be dropped before merge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Signed-off-by: Saravana Periyasamy <saperiyasamy@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview:
Mitigates intermittent deploy/operator CI failures caused by stale Pod readiness synchronization
between vCluster and the host cluster. The operator webhook is healthy, but its endpoint can be
excluded from Service routing until the stale readiness state recovers.
Details:
Adds a configurable webhook.service.publishNotReadyAddresses Helm value, defaulting to false.
Enables the value only for short-lived vCluster CI environments.
Verifies that both the virtual and host webhook Services preserve the override.
Keeps real v1alpha1 and v1beta1 admission requests as the final health check.
Adds focused diagnostics for Pods, Services, Endpoints, EndpointSlices, operator logs, and
vCluster syncer logs.
Adds Helm tests covering the production default and CI override.
Validation:
Where should the reviewer start?
Start with:
.github/actions/setup-dynamo-operator/action.yml for the CI-only override, verification, and
diagnostics.
deploy/helm/charts/platform/components/operator/templates/webhook-service.yaml for the
rendered Kubernetes Service setting.
deploy/helm/charts/platform/tests/webhook_service_test.yaml for default-safety and override
coverage.
Related Issues
🚫 This PR is NOT linked to an issue:
Summary by CodeRabbit