fix(sglang): stop the elastic-EP scale-up worker crash-looping at startup

[glamr-agent]

Source issue: DYN-4325.

Overview:

SGLang will not start an elastic-EP deployment that can scale up unless both its decode and its prefill CUDA graph are disabled. A CUDA graph is a replayed, fixed-shape execution plan, and elastic EP changes that shape at runtime, so SGLang asserts the two are off during argument resolution. The elastic-EP test manifest passed no CUDA-graph flag at all, so SGLangDecodeWorker hit that assertion and crash-looped before serving anything.

Details:

tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml now passes --cuda-graph-backend-decode disabled and --cuda-graph-backend-prefill disabled in the worker's argument list, and its header REQUIREMENTS comment records the precondition. --moe-a2a-backend nixl already disabled the prefill side as a side effect, but nothing disabled the decode side, so both phases are now set explicitly. run_sglang_elastic_ep_scale_up_test.sh gets the same two flags in launch_joiner, which its own comment already requires to match the manifest.

These per-phase flags are used instead of --disable-cuda-graph, which is deprecated in the pinned sglang==0.5.19 and sits below the per-phase flags in precedence, so a later tuning flag would silently bring the crash back. The SGLang recipes under recipes/glm-5.2/sglang/ already spell it this way.

One gap stays open: launch_joiner still passes no --node-rank 1, which SGLang asserts separately for --elastic-ep-join-mode scale. That is an independent defect, so this change alone does not make the scale-up script run end to end.

Where should the reviewer start?

tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml, to confirm the two flags land inside spec.services.SGLangDecodeWorker.extraPodSpec.mainContainer.args and not in a sibling mapping.

Validation

This manifest needs eight 80 GiB GPUs and RDMA on a Kubernetes cluster, which was not available, and no CI job covers tests/fault_tolerance/deploy/*. Instead: pre-commit run --files <the two changed files> --hook-stage manual passed, and loading the YAML with yaml.safe_load confirmed the flags sit in the worker's args list as adjacent flag/value pairs. The assertion and the flag precedence were read from the sglang v0.5.19 sources named above.

Related Issues

🚫 This PR is NOT linked to an issue:

• Confirmed — no related issue (an internal NVIDIA ticket tracks this work; it is linked at the top of this description)

Summary by CodeRabbit

• Bug Fixes

  • Improved reliability of elastic expert-parallel scale-up operations by ensuring newly joined workers use compatible runtime settings, with CUDA graph backends disabled for decoding and prefilling.

• Documentation

  • Clarified that CUDA graph acceleration must be disabled for both decoding and prefilling during elastic scale-up scenarios.
  • Updated deployment guidance to reflect the required configuration for successful worker integration and consistent runtime behavior.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi glamr-agent! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[glamr-agent]

Automated evidence record — validation complete

Validation status: complete

Evidence summary: [3/3 validated]

Automated review assessment: sound. This is an advisory machine-generated review, not a human approval, and it does not replace maintainer review.

Validation result: complete — pass.

Evidence audit: complete [3/3 validated] — the command report below comes from recorded runs.

Commands and results [3/3 validated]

Generated from the commands recorded during this run.

Check 1

Builds the changed Dynamo source and confirms that Python can import its compiled extension.

Result: Passed (exit 0)

Command:

python3 -c 'import dynamo, dynamo._core; print('\''dynamo and dynamo._core import OK'\'')'

Check 2

Checks the changed files with the repository's fast lint and formatting commands.

Result: Passed (exit 0)

Command:

pre-commit run --files tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml tests/fault_tolerance/deploy/templates/sglang/run_sglang_elastic_ep_scale_up_test.sh --hook-stage manual

Check 3

Inspects the changed code when the claim cannot be tested with a local command.

Result: Passed (exit 0)

Command:

/scratch/sglang-arg-check/venv/bin/python /scratch/sglang-arg-check/sglang_elastic_ep_arg_resolution_check.py

[glamr-agent]

No description provided.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8f498086-3be6-4c02-b3d3-b67102df822b

📥 Commits

Reviewing files that changed from the base of the PR and between 7650b76 and fcf86d9.

📒 Files selected for processing (2)

• tests/fault_tolerance/deploy/templates/sglang/run_sglang_elastic_ep_scale_up_test.sh
• tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

---

Walkthrough

The SGLang elastic-EP scale-up templates disable decode and prefill CUDA graph backends. The deployment documentation records this requirement.

Changes

SGLang elastic-EP scale-up

Layer / File(s)	Summary
CUDA graph backend configuration tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml, tests/fault_tolerance/deploy/templates/sglang/run_sglang_elastic_ep_scale_up_test.sh	The deployment documentation, worker arguments, and joiner command disable decode and prefill CUDA graph backends.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Merge Risk: ⚪ Minimal · up to fcf86

Elastic-EP scale-up workers and joiners now disable CUDA graphs to avoid startup crash-looping. No concrete current-head merge risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description explains the cause, implementation, validation, reviewer starting point, and remaining limitation. However, the required Related Issues section is inconsistent because it provides sour…	Update the Related Issues section to link the issue using the required format, such as "Relates to #DYN-4325" if applicable, or provide the repository issue number. Remove the contradictory no-related-issue option and delete the unused alte…

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the primary change: preventing the SGLang elastic-EP scale-up worker from crash-looping at startup.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description explains the cause, implementation, validation, reviewer starting point, and remaining limitation. However, the required Related Issues section is inconsistent because it provides source issue DYN-4325 while selecting the no-related-issue option.

Resolution

Update the Related Issues section to link the issue using the required format, such as "Relates to #DYN-4325" if applicable, or provide the repository issue number. Remove the contradictory no-related-issue option and delete the unused alternative path.

• Fix all pre-merge checks with AI

---

_{Comment @coderabbitai help to get the list of available commands.}

[glamr-agent]

CI reached a terminal state on head fcf86d9 with no failing and no pending checks: pre-commit, codeowners, DCO, copyright-checks, lychee, changed-files, pre-merge-status-check, and CodeRabbit all passed. The heavier build and test jobs — rust-tests, rust-clippy, operator, Recipe Check, and the docs jobs — did not run: this is a fork pull request, so they stay gated until a maintainer comments /ok to test fcf86d9 with the current head SHA. Until that happens the passing result covers the lint, metadata, and policy checks only.

[glamr-agent]

Addressed the review note on tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml. The inline comment above the CUDA-graph arguments restated the entry already present in the file's REQUIREMENTS header, so it is removed and the header entry is now the single statement of that precondition. The arguments themselves are unchanged: the worker still passes --cuda-graph-backend-decode disabled and --cuda-graph-backend-prefill disabled.

Commit fcf86d9, fix(sglang): drop duplicate CUDA-graph note in elastic-EP template.

Checks run on the changed file:

pre-commit run --files tests/fault_tolerance/deploy/templates/sglang/sglang_elastic_ep.yaml

Every applicable hook passed, including check yaml, codespell, check for merge conflicts, mixed line ending, and trim trailing whitespace. The Python, C++, and docs hooks reported no files to check for this path.

[glamr-agent]

/devin review

@coderabbitai full review

[coderabbitai]

@glamr-agent I will perform a full review of PR #14568.

✅ Action performed

Full review finished.

[dagil-nvidia]

/ok to test fcf86d9

[yunzhoul-nv]

/ok to test b1b68a2

[yunzhoul-nv]

/ok to test ae3cddd