ci(deploy-test): align vCluster Kubernetes with host and remove readiness workaround

[sara4dev]

Summary

• Derive the vCluster guest Kubernetes version from the host API at setup time, normalizing provider-suffixed versions to the upstream vMAJOR.MINOR.PATCH image tag.
• Remove the temporary publishNotReadyAddresses overrides for the Dynamo and Grove webhook Services, including the chart value and its dedicated unit test.
• Keep the readiness gates, admission smoke tests, Grove health probes, and failure diagnostics added by fix(ci): tolerate stale vcluster webhook readiness #12278 and fix(ci): harden Grove webhook readiness in vCluster #12368.
• Preserve fix(ci): remove Grove mutation assertion #12399's removal of the brittle Grove defaulting-mutation assertion.

Fixes #12266.

Root cause

The deploy-test vCluster guest was pinned to Kubernetes v1.32.13 while the host had moved to v1.34.9. That skew triggers loft-sh/vcluster#3578: the syncer can fail to write Pod status because status.qosClass is immutable, leaving guest Pods stale even when their host Pods are running.

The old static pin could drift again on the next host upgrade. This change reads the host's serverVersion.gitVersion and uses its upstream semantic version for the vCluster control-plane image.

Validation

• Rebased onto 4830f0c7ad, which includes fix(ci): remove Grove mutation assertion #12399.
• DCO-signed commit; signature verified locally.
• Targeted pre-commit hooks passed, including YAML, whitespace, conflict, and codespell checks.
• Composite action parses successfully with all 18 steps intact and no stale vcluster_k8s_version input.
• Version normalization verified for upstream, EKS-suffixed, and build-suffixed Kubernetes versions.
• Operator Helm chart lints and renders successfully; the rendered Service omits publishNotReadyAddresses.
• Full CI passed on 187aacbf02: deploy-operator, both operator jobs, and all eight vLLM/SGLang/TRT-LLM deploy variants are green.

The first full-CI attempt hit the self-hosted runner's 30-minute cutoff in an unrelated TRT-LLM parallel GPU test; the same job failed concurrently on PR #12363. A failed-jobs rerun passed without code changes.

The full platform Helm dependency build could not complete locally because the external NATS chart download stalled; the component chart used by this change was linted and rendered directly.

Out of scope

• The shared-vCluster bootstrap race between parallel matrix jobs.
• The 1800-second deployment wait running under a 1200-second pytest timeout.

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

Walkthrough

The setup action now defaults vCluster Kubernetes versions to v1.34.9, requires alignment with the host cluster version, and documents synchronization and pod-readiness failures caused by unsupported guest/host version skew.

Changes

vCluster Version Alignment

Layer / File(s)	Summary
Version input and creation guidance .github/actions/setup-dynamo-operator/action.yml	The vcluster_k8s_version default and description now target host-version alignment, while the vCluster creation comments document syncer and readiness issues caused by version skew and kr8s support constraints.

Estimated code review effort: 1 (Trivial) | ~3 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	It doesn't implement the #4037 pod-status reconciliation fixes; it only changes the deploy-test vCluster version pin, so the linked issue's core requirements remain unmet.	Add the qosClass and observedGeneration sync fixes from #4037, or retarget the PR to the deploy-test workaround and link the proper issue.
Description check	⚠️ Warning	The PR has summary, root cause, validation, and out-of-scope notes, but it omits required template sections like Overview, Details, reviewer start, and the issue-link format.	Add the missing template sections, especially Where should reviewer start? and a properly formatted Related Issues section with the linked issue.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The only code change is the vCluster version pin and related comment update, which is directly aligned with the stated objective.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title clearly matches the main change: aligning vCluster Kubernetes versions and removing the readiness workaround.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/actions/setup-dynamo-operator/action.yml:
- Around line 50-52: Remove the default value from the Kubernetes version input
in the setup-dynamo-operator action, and update every caller to pass an explicit
supported version aligned with its host cluster. Keep the input optional only if
the action’s existing validation permits omission; otherwise make it required so
no call site silently inherits a version.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 42f6b3a2-b3d7-4b69-ac65-763bbcf86841

📥 Commits

Reviewing files that changed from the base of the PR and between 8e8ce2c and 8631c71.

📒 Files selected for processing (1)

• .github/actions/setup-dynamo-operator/action.yml

[datadog-official]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 42.08% (-5.53%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 187aacb | Docs | Datadog PR Page | Give us feedback!}

[github-actions]

🌿 Fern Docs Preview: https://nvidia-preview-019fb04b-de30-75ca-b27e-6b0d473c8ef8.docs.buildwithfern.com/dynamo/dev

[dagil-nvidia]

Approving. Green across 81 checks, and this removes the CI workaround rather than adding one.

The chart-surface removal is safe: dynamo-operator.webhook.service.publishNotReadyAddresses was introduced yesterday in #12278 and is not in release/1.3.0, so it never shipped and nothing can depend on it. Better to take this before the v1.4.0 cut than to ship a production chart value that exists only to paper over a CI environment bug.

One follow-up, not a blocker: the previous code carried an explicit constraint that the vCluster guest version must be supported by the kr8s Python library, deliberately independent of the host. Deriving it from the host couples them, so a future host upgrade past kr8s support will surface as confusing deploy-test failures rather than an unsupported-version error. Worth confirming kr8s covers 1.34, and considering an upper bound or supported-version allowlist.