Repository navigation
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for the contribution!
中文感谢你的贡献！
|
zijiexia
added
full-sweep-fail-fast
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Add disaggregated SGLang behind Dynamo for MiniMax-M3.1-Flash AgentX on GB200 with lmsysorg/sglang:nightly-cu134-20261030-e371881 (Dynamo 1.7.0.dev20261030, Mooncake transfer, minimax-msa kernels)." | ||
| - "2 DEP4 prefill workers with a DEP8 decode at conc 224 and 256, or 3 DEP4 prefill workers with a DEP16 decode at conc 320 and 360; MegaMoE, DSpark block 7 with fp8 draft KV, Dynamo KV router with fastokens for prefill and least-loaded routing across decode DP ranks." | ||
| - "Prefill: shortest-prefill-first, chunk 8192, mem-fraction-static 0.85, HiCache 180 GB per rank write_back." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX |
There was a problem hiding this comment.
🔴 The changelog check will fail on this PR, so the new config never gets scheduled. Its pr-link ends in pull/XXXX, with four X's. The validator only accepts XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX, or the PR's real URL (validate_perf_changelog.py:21-24). Fix: set pr-link to this PR's real URL, or to the exact .../pull/XXX placeholder.
Why this was flagged
The new entry at inferencex-e2e/perf-changelog.yaml:11750 sets pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXX. On PR runs, validate_added_pr_link (inferencex-e2e/infx/workflows/validate_perf_changelog.py:105-116) accepts only a link in PR_LINK_PLACEHOLDERS ("XXX" or "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX") or the PR's own URL. Four X's matches none of these, so it raises "new PR entry must use ... or an XXX placeholder". On main, CANONICAL_PR_LINK needs pull/\d+ and rejects it too. The changelog validation fails, so the sweep for minimaxm31flash-fp4-gb200-dynamo-sglang-agentic-dspark-disagg is never scheduled and the PR cannot get the /use <run_id> acceptance it needs.
Verification: Trigger: the PR goes through merge_with_reuse. The entry at inferencex-e2e/perf-changelog.yaml:11750 uses pull/XXXX, which matches neither placeholder (validate_perf_changelog.py:21-24). merge_with_reuse.py:688 calls canonicalize_changelog(pr, repo) and does not catch the exception, so the merge aborts. The PR-run changelog check does not call this validator.
| multinode: true | ||
| disagg: true | ||
| srt-recipe-dir: minimaxm31flash/sglang/gb200-fp4/agentx | ||
| scenarios: |
There was a problem hiding this comment.
🟡 (optional) Published results for this config will claim 901 GB of host KV per node, while the recipe only pins 720 GB. dram-utilization: 1.0 makes matrix generation record the whole gb200-nv node share (860_160 MiB, about 901 GB) as total-cpu-dram-gb. The prefill recipe pins hicache-size: 180 on 4 ranks, which is 720 GB. Fix: set dram-utilization: 0.80, which resolves to 721 GB. The single-node sibling minimaxm31flash-fp4-gb200-sglang-agentic-dspark uses that value for the same 180 GB-per-rank pool.
Why this was flagged
agentic_dram_offload_gb (infx/matrix/generate.py:475-518) computes the prefill share as available-cpu-dram-mib 860_160 (runners.yaml:572) × utilization × 4/4 GPUs. With 1.0 that comes to 901 GB. The value reaches TOTAL_CPU_DRAM_GB through infx/srt_slurm/generate.py:59. It is published as allocated_cpu_dram_gb (infx/results/agentic/init.py:205). The decode side also gets 901 GB as decode_allocated_cpu_dram_gb, even though decode runs no HiCache. The recipe really allocates 180 GB × 4 ranks = 720 GB per prefill node; its own comment says "720 of the node's ~900 GB host allowance". The same-cluster, same-pool sibling uses dram-utilization 0.80. Its recipe records TOTAL_CPU_DRAM_GB '721' (benchmarks/single_node/.../minimaxm31flash/sglang/gb200-fp4-mtp/agentic.yaml:148). So results with identical host pools would report different DRAM budgets, and this one overstates its pool by about 25%. Nothing checks the recorded value against hicache-size, because the recipe never reads TOTAL_CPU_DRAM_GB.
Verification: nit. This happens on every DRAM-offload point of the new config. In agentic_dram_offload_gb, share_gb returns 860160 MiB × 1.0 × 4/4, which is about 901 GB. infx/results/agentic/__init__.py:205 publishes this as allocated_cpu_dram_gb. The recipe really pins hicache-size: 180 on 4 ranks per prefill node, which is 720 GB. Nothing fails at runtime.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=41271615328 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=41271615328 |
…-sglang-pd # Conflicts: # inferencex-e2e/perf-changelog.yaml
…-sglang-pd # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
/use 41271615328 |
|
@nvpohanh staging run 41271615328. Existing staged runs will be preserved; a completion link will be posted here. |
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
|
[by Claude Code] sgl-project/sglang#47929 has been merged. This PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
|
[by Claude Code] sgl-project/sglang#48392 has been merged. This PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
|
[by Claude Code] sgl-project/sglang#48634 has been merged. This PR's AgentX configs ( The commands mirror the launch measured in run 41271615328, pinned by this PR's |
Update config.
中文
更新配置。
AI model disclosure
Prepared with Claude Opus 5.5 (claude-opus-5-5).
🤖 Generated with Claude Code