Repository navigation
feat(agentx): add Muse Spark Open GB200 SGLang DFlash2 AgentX config - #4140
Conversation
Serve meta-models/Muse-Spark-Open-816B-A42B-NVFP4 on one GB200 node at TP4/EP4 from the staged Lustre checkpoint with the DFlash2 draft, and let the srt connector derive DFlash's draft length from its verify width.
|
Thanks for the contribution!
中文感谢你的贡献！
|
cquil11
added
the
non-canary-full-sweep-enabled
|
Claude finished @cquil11's task in 1m 3s —— View job Review: Muse Spark Open GB200 SGLang DFlash2 AgentX
🟡 1 issue (2 inline comments). The changelog commit replaced the
The validator accepts pr-link-only corrections, so CI won't flag this, but those results would be credited to this PR. Revert the two lines or point them at their real PRs. Everything else checks out:
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Topology: 1 prefill node (TP8) + 1 decode node (TP8), each 8xB200 exclusive; TileRT decode is bs=1 only so conc-list is a single point [1], ISL 1k/8k OSL 1k" | ||
| - "Runner: launch_b200-dgxc.sh tilert early-return branch (zero impact on the dynamo path); tilert_utils/submit.sh issues two srun --ntasks=1, one per role, because prefill and decode need different container images; roles are dispatched by the TILERT_ROLE it exports, and torn down across nodes via a sentinel file on the shared /workspace" | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4140 |
There was a problem hiding this comment.
🟡 WARNING: This entry's XXX placeholder now points to #4140. The entry is the TileRT glm5.1-fp8-b200-tilert entry from #2533 (commit 077d483), not Muse Spark. It looks like a blanket XXX→4140 replace in commit a107f97 caught it.
Why it matters: The changelog is the historical record, and ingest/recovery tooling uses pr-link to attribute results. These TileRT results would be credited to this PR.
Fix: Revert this line, or correct it to the real PR:
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4140 | |
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2533 |
| - "Apply the point-specific prefill-decode interval, prefill chunking, admission, memory and SWA prefix-tail settings from the single-node recipe; all points use the default fcfs schedule policy." | ||
| - "No data-type or precision change to the DSpark draft: the bundled draft of deepseek-ai/DeepSeek-V4.1-Flash runs as shipped with block size 5." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4140 |
There was a problem hiding this comment.
🟡 WARNING: Same problem as the TileRT entry. This is the dsv41flash-fp4-gb200-sglang-agentic-dspark refresh from #3927 (commit 2b97216), not this PR.
Why it matters: The DSV4.1-Flash GB200 SGLang results would be attributed to #4140.
Fix:
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4140 | |
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/3927 |
There was a problem hiding this comment.
I checked two more things beyond the inline finding, and neither is a problem:
- Setting
single-node-models: stagedforgb200-nvchanges only this config. The other gb200-nv single-node configs, the two dsv41flash ones, serve DeepSeek-V4.1-Flash, and the gb200-nvmodels.entrieslist has no entry for it, so they still load fromhf:. - The changelog's "AL 4.05" is right: 8 verify tokens minus 1 gives
num_speculative_tokens7, and the committedmusespark_dflash.yamlmaps 7 to 4.05. The 3.4 value is only in the test fixture.
Extended reasoning...
The change adds a single-node GB200 SGLang DFlash2 agentic config: recipe, master config, a runners.yaml staged entry, a DFlash branch in synthetic_acceptance, a test and a changelog block. In the cluster record, single_node_model_path uses a staged path only when checkpoint() finds an entry whose name matches the model basename, and no other gb200-nv single-node model has one. The real golden curve's key 7 is 4.05, which matches what the PR description says.
| - "Apply the point-specific prefill-decode interval, prefill chunking, admission, memory and SWA prefix-tail settings from the single-node recipe; all points use the default fcfs schedule policy." | ||
| - "No data-type or precision change to the DSpark draft: the bundled draft of deepseek-ai/DeepSeek-V4.1-Flash runs as shipped with block size 5." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4140 |
There was a problem hiding this comment.
🔴 Changelog validation in CI will fail for this PR, and two older entries would point at #4140 although it did not make those changes. Lines 5598 and 9727 change historical pr-link: .../pull/XXX lines in the same diff that appends the new block. compare_entries rejects that mix with "do not mix historical pr-link corrections with new changelog entries". Fix: restore both historical pr-link lines byte-for-byte so the only change is the appended musespark block. Any XXX placeholder fixes belong in their own correction-only change that uses each entry's real PR number.
Why this was flagged
The diff appends a new entry at the end of inferencex-e2e/perf-changelog.yaml. It also changes two existing entries' pr-link from pull/XXX to pull/4140: the TileRT entry at line 5598 and the dsv41flash-fp4-gb200-sglang-agentic-dspark entry at line 9727. In infx/workflows/validate_perf_changelog.py:compare_entries (lines 184-190), each changed pr-link counts as a correction. When there are both corrections and additions, it raises ChangelogValidationError("do not mix historical pr-link corrections with new changelog entries"). validate_raw_change (lines 205-210) would reject the file too, because head_raw no longer starts with base_raw. That breaks AGENTS.md:140 ("append-only and byte-sensitive ... Preserve all existing bytes"). Even if this got past CI, the TileRT and DSpark entries would now link to PR #4140, which did not make them. #4140 would then show up in three blocks, against the one-block-per-PR rule at AGENTS.md:141. On the base branch those entries keep their XXX placeholders and the validator passes.
Verification: At inferencex-e2e/perf-changelog.yaml lines 5598 and 9727 the diff changes pr-link: .../pull/XXX to .../pull/4140 in existing entries and also appends a new musespark block. In inferencex-e2e/infx/workflows/validate_perf_changelog.py lines 186-190, compare_entries raises "do not mix historical pr-link corrections with new changelog entries" when both are present.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38834513220 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38834513220 |
|
/use 38834513220 |
|
@functionstackx staged run 38834513220. This run remains available across future |
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX config ( The commands mirror the launch measured in run 38834513220, pinned by this PR's |
|
[by Claude Code] sgl-project/sglang#44654 has been merged. This PR's AgentX config ( The commands mirror the launch measured in run 38834513220, pinned by this PR's |
3 participants
Adds Muse Spark Open (
meta-models/Muse-Spark-Open-816B-A42B-NVFP4) AgentX on one GB200 node: SGLanglmsysorg/sglang:dev-muse-spark-open(digest-pinned), TP4/EP4, DFlash2 draftRadixArk/Muse-Spark-Open-DFlash2with 8 verify tokens, HiCache host KV offload (120 GB per rank), concurrency 1 to 32.mem-fraction-staticis 0.88, which gives a 1.7M-token KV pool.models.entries+single-node-models: stagedforgb200-nv; no other gb200 single-node model has a staged entry, so their Hub path is unchanged). The AgentX client reads the tokenizer from/model, since the gated Hub repo is not readable with the CI token.musespark_dflashcurve. The srt connector now derives DFlash's draft length from--speculative-num-draft-tokensminus the verify token (DFlash leavesspeculative-num-stepsat 1), so 8 verify tokens select AL 4.05; the server reports accept len ~4.05. The connector and single-nodedflashallowance are the same hunks as [Klaud Cold] Add Muse Spark Open MXFP4 MI355X SGLang DFlash2 AgentX recipe / [Klaud Cold] 新增 Muse Spark Open MXFP4 MI355X SGLang DFlash2 AgentX 配方 #4127.Validation
Sweep 38834513220 on
5b0708723: 10/10 AgentX points and the GSM8K eval succeeded (GSM8K strict 0.976). All 10 points are on the throughput/interactivity frontier.