Skip to content

[Docs] Add GB300 NVFP4 multi-node GLM-5.2 recipe - #42829

Closed
Ankur-singh wants to merge 2 commits into
sgl-project:mainfrom
Ankur-singh:docs/glm52-gb300-multinode
Closed

Ankur-singh wants to merge 2 commits into
sgl-project:mainfrom
Ankur-singh:docs/glm52-gb300-multinode

Conversation

@Ankur-singh

@Ankur-singh Ankur-singh commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

The GLM-5.2 Deployment selector has no GB300/NVFP4/Low-Latency multi-node cell, so it disables Multi-Nodes. This adds a compact recipe for one TP8 worker spanning two four-GPU GB300 nodes.

Modifications

  • Enable GB300 → NVFP4 → Low-Latency → Multi-Nodes with 12 serving flags and two NCCL fabric environment variables. The renderer supplies node count, node rank, and head-node address. Keep NVFP4, EAGLE 5-1-6, concurrency 8, and memory fraction 0.8 explicit.
  • Put the four HiCache flags and their 540 GB-per-node host-memory requirement in a collapsed Optional HiCache section. Omit redundant EP1, loading/observability options, and benchmark-specific graph, prefill, context, and environment overrides from the main command.
  • Document fabric and node setup. Keep the recipe Not Verified, with no borrowed benchmark results; FP8 multi-node remains unavailable.

Source and qualification

Derived from InferenceX #3659's TP8 C4 recipe at e79239ba, tested in run 36997573694, attempt 2.

That sweep used Dynamo and lmsysorg/sglang:nightly-dev-20260914-4358a161. This simplified standalone command uses the cookbook's existing GB300 image selection (lmsysorg/sglang:latest) and does not reproduce the full benchmark configuration. The standalone adaptation and current image have not been tested on GPUs.

Related #42726 covers broader Dynamo/disaggregated recipes. This patch addresses the existing selector and does not claim to complete all of #3659's recipe requirements.

Accuracy Tests

  • Cookbook config check, Mintlify build validation, internal links/anchors/redirects, and whitespace check against the PR base — passed.
  • Playwright: Python/Docker output, Single Node → Multi-Nodes, TP8/MTP settings, Playground inheritance without default HiCache, and optional HiCache accordion — passed. No browser console errors.

GPU startup and model evaluation were not run. The recipe remains unverified.

Speed Tests and Profiling

Not run; no benchmark numbers are added.

Checklist

  • Validate documentation, config, links, and rendered controls.
  • Keep the change declarative; shared rendering engines are unchanged.
  • Validate standalone serving on GPUs before marking it verified.

Prepared with Codex (GPT-6); the exact runtime model version was not exposed.


CI States

Latest PR Test (Base): ✅ Run #37545675299
Latest PR Test (Extra): ❌ Run #37545675082
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 6, 2026
@Ankur-singh

Copy link
Copy Markdown
Contributor Author

@faradawn Can you please help review this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant