[shyeh25]

Summary

Update the Muse Spark Open B200 SGLang AgentX curve to lmsysorg/sglang:nightly-dev-20261021-c1c287a4, set SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0 in the base serving environment, and add TP8/EP8 concurrency 64:

• TP8/EP8, GPU-resident KV: concurrency 1, 2, 4, 6, 8, 10
• TP8/EP8, DRAM HiCache: concurrency 56, 64
• TP4/EP4 on 4 GPUs, DRAM HiCache: concurrency 8, 10, 16, 24

Per-point changes:

• TP8 concurrency 56:
  • 32768-token chunked prefill and max prefill tokens
  • prefill-decode interval 48 → 64
  • Python radix tree core inherited from the base (the SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND: null override is removed)
• TP8 concurrency 64 (new), starting from the previous concurrency 56 settings:
  • request admission and decode graph batch 64
  • 32768-token chunked prefill and max prefill tokens
  • prefill-decode interval 48 and the Python radix tree core
• TP4 concurrency 24: memory fraction 0.95 → 0.98

TP8 concurrency 1, 2, 4, 6, 8, 10 and TP4 concurrency 8, 10, 16 keep their per-point settings; they only pick up the new image and the base environment variable. The checkpoint and DFlash2 draft settings are unchanged.

AI model disclosure

• Model/version: claude-opus-5-5 (the identifier exposed by the runtime).
• Role: ran the tuning measurements, generated the recipe, master-config and perf-changelog changes from the measured runs, checked them locally against the measured server configurations, and drafted this PR description.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

将 Muse Spark Open B200 SGLang AgentX 曲线更新到 lmsysorg/sglang:nightly-dev-20261021-c1c287a4，在 base 的服务环境变量中设置 SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0，并新增 TP8/EP8 并发 64：

• TP8/EP8，KV 留在 GPU：并发 1、2、4、6、8、10
• TP8/EP8，DRAM HiCache：并发 56、64
• TP4/EP4（4 张 GPU），DRAM HiCache：并发 8、10、16、24

各点改动：

• TP8 并发 56：
  • chunked prefill 与 max prefill tokens 设为 32768 token
  • prefill-decode interval 48 → 64
  • 改用 base 继承的 Python radix tree core（移除 SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND: null）
• TP8 并发 64（新增），以原并发 56 的设置为基础：
  • 请求数上限与 decode graph batch 为 64
  • chunked prefill 与 max prefill tokens 设为 32768 token
  • 保留 prefill-decode interval 48 与 Python radix tree core
• TP4 并发 24：memory fraction 0.95 → 0.98

TP8 并发 1、2、4、6、8、10 与 TP4 并发 8、10、16 的各点设置不变，只随之更新镜像与 base 环境变量。checkpoint 与 DFlash2 draft 设置不变。

AI 模型使用说明：运行环境提供的模型标识为 claude-opus-5-5。工作内容包括执行调优量测、根据实测结果生成配方、主配置和性能变更记录的改动、在本地与实测的服务配置逐项核对，以及起草本 PR 说明。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Code review found no issues

No high-confidence issues detected in this change.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=39756010759

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=39756010759

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX config (musespark-fp4-b200-sglang-agentic-dflash) will happen in sgl-project/sglang#46005, in the Agentic Long-Context Deployment section of the Muse Spark Open cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (11358f484) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#46005 has been merged. This PR's AgentX config (musespark-fp4-b200-sglang-agentic-dflash) is now in the Agentic Long-Context Deployment section of the SGLang Muse Spark Open cookbook: https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (11358f484) as "in progress". A later cookbook PR switches it to the measured run once a run is pinned or this PR merges.

[nvpohanh]

/use 39756010759

[github-actions]

@nvpohanh staging run 39756010759. Existing staged runs will be preserved; a completion link will be posted here.

[nvpohanh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• For affected Dynamo TRT-LLM benchmarks, verified that iteration/KV-cache metrics are collected and published under the pinned Dynamo implementation and effective engine settings. Resolve the installed Dynamo version separately from the container tag; use supported metrics controls without requiring KV events. Record the version and configuration evidence in the additional detail section. See verifier Check 17. N/A for unaffected serving paths or frameworks without these Dynamo settings.
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
• Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

• Scope: one config, musespark-fp4-b200-sglang-agentic-dflash (single-node aggregated SGLang on B200 with DFlash2), agentic-coding only. Changed files: musespark/sglang/b200-fp4-mtp/agentic.yaml (base env SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION: '0'; TP8 c56 gets 32768 chunked prefill and max prefill, interval 48 → 64, and drops the SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND: null override; new override_tp8_c64; TP4 c24 mem-fraction-static 0.95 → 0.98), nvidia-master.yaml (image lmsysorg/sglang:nightly-dev-20261021-c1c287a4; the TP8 HiCache row becomes conc-list: [56, 64]), and one appended perf-changelog.yaml entry. Assessed head: 11358f4843ec3e086014d3bba1a102e400944a59, the only commit in the PR.
• PR validation: Run Sweep 39756010759 (attempt 1) on head 11358f48. All 12 agentic / jobs succeeded: TP8 c1, 2, 4, 6, 8 and 10 without offload, TP8 c56 and c64 with HiCache, and TP4 c8, 10, 16 and 24 with HiCache. The agentic eval / job and check-changelog also succeeded. Fixed-sequence and multi-node lanes were skipped because this PR has no such configs. The run is pinned with /use 39756010759.
• Evals: the agentic eval / job ran the new TP8 c64 recipe on the same image, lmsysorg/sglang:nightly-dev-20261021-c1c287a4, with SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0. gsm8k em_strict = 0.9757 (± 0.0042, n=1319), above the 0.90 default threshold in infx/evals/thresholds.yaml, which has no musespark override. Artifact: eval_results_all from run 39756010759.
• Chat template: AgentX replays send requests to POST /v1/chat/completions (TP8 c64 benchmark.out). Every server runs with chat_template=/model/chat_template.jinja.
• Draft precision (served as shipped): the draft is RadixArk/Muse-Spark-Open-DFlash2 (speculative_draft_model_revision=main, snapshot 46c90f412549c472941a68382d91a99e5e6372a0). It ships as BF16: config.json has dtype: bfloat16 and no quantization_config, and the Hub safetensors metadata reports 4,887,479,552 BF16 parameters. No point sets a draft quantization, dtype, revision or draft-KV override. The server args on all 12 points show _speculative_draft_quantization_explicitly_set=False, speculative_draft_kv_cache_dtype=None and json_model_override_args='{}'.
  • The server args show speculative_draft_model_quantization='modelopt_fp4'. This comes from the pinned image's default inheritance of the target's --quantization, not from a submission setting. At sglang c1c287a4, a draft checkpoint without quant metadata goes through ModelOptFp4Config.for_online_weight_quantization (weight_utils.py). Its get_quant_method (nvfp4_online.py) returns UnquantizedLinearMethod for every LinearBase layer when the checkpoint is not FP8-serialized, and quantizes only FusedMoE. DFlash2DraftModel (models/dflash.py) has no FusedMoE; its MLP is the dense DFlashMLP.
  • The logs match: on all 12 points the draft loads as Load weight end … type=DFlash2DraftModel with no quant tag, at 2.60–2.64 GB per GPU on TP8 and 3.61–3.63 GB per GPU on TP4. The target logs quant=modelopt_fp4, quant_algo=NVFP4. Effective draft weight precision: BF16 = shipped precision, at every point.
  • The new SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0 does not reach the draft. At c1c287a4 it is read only by ModelOptFp4Config for serialized NVFP4 checkpoints (modelopt_quant.py) and by the MoE argument hook. It selects the per-tensor rather than per-token FP32 activation scales of the target's NVFP4 MoE. The draft's online config has no serialized NVFP4 layers and no FusedMoE, so the draft does not use the setting. The pinned muse_spark_hook.py would otherwise default it to True for Muse Spark. The explicit 0 keeps that default off, and the server logs confirm it is not auto-applied. This is a target-only activation-scale choice, and the eval above covers it.
  • Draft KV cache: TP8 c1–c10 keep the base speculative-draft-attention-backend: fa4, and the image forces BF16 draft KV for those points (DFLASH fa4 draft: overriding KV cache dtype torch.float8_e4m3fn -> torch.bfloat16). TP8 c56 and c64 and all TP4 points unset that override. The draft then follows the pinned default, the target backend (trtllm_mha), and inherits the target's upstream-supported fp8_e4m3 KV dtype (Post-capture KV sizing: draft KV cache allocated. dtype: torch.float8_e4m3fn). The FP8 KV dtype is applied consistently to target and draft, and the draft weights and activations stay BF16.
  • SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled. It does not appear in the recipe, the master config, the generated config.yaml/sbatch_script.sh, or the server logs of any of the 12 points. Its pinned default is EnvBool(False) (environ.py at c1c287a4), and DFlash2 is a dense draft with no NextN MoE.
• Golden AL: the connector (infx/srt_slurm/synthetic_acceptance.py) maps DFlash speculative-num-draft-tokens: 8 to num_speculative_tokens = 7. musespark_dflash.yaml gives muse-spark-open-816b-a42b-nvfp4 / thinking_on / 7 = 4.05. The submissions of all 12 points show SGLANG_SIMULATE_ACC_LEN=4.05, SGLANG_SIMULATE_ACC_METHOD=match-expected and SGLANG_SIMULATE_ACC_TOKEN_MODE=real-draft-token. The recipe hard-codes none of them. The TP8 c64 server log reports a mean accept length of 4.05 over 2118 decode log lines.
• MODELS.md: musespark (Muse Spark Open 816B-A42B, added 2026-10-15) lists Agentic coding with the SGLang DFlash2 draft as active, with no deprecated scenarios (review date 2026-10-23).
• No architecture hacks: there is no --hf-overrides or json-model-override-args (effective '{}'), and no layer, expert or indexer skipping. The changes are an image bump, the target NVFP4 MoE activation-scale mode, chunked prefill and max prefill size, the prefill-decode interval, the radix-tree core backend, memory fraction, and the new c64 point's admission and CUDA-graph batch.
• Dynamo TRT-LLM metrics: N/A. This is a direct SGLang deployment with no Dynamo TRT-LLM serving path.
• Upstream image / engine order: lmsysorg/sglang:nightly-dev-20261021-c1c287a4 comes from the upstream lmsysorg org (sglang commit c1c287a4). This is an SGLang submission, so the engine-first ordering is satisfied.
• Upstream recipe: sgl-project/sglang#46005 is MERGED (2026-10-23T08:04:02Z, merge commit 02d24451). It is published at https://docs.sglang.io/cookbook/autoregressive/Meta/MuseSparkOpen#agentic-long-context-deployment (docs/src/snippets/agentx/meta-models/muse-spark-open.jsx, B200 entries #0 "Agg TP8·EP8" and #2 "Agg TP4·EP4", provenance sha 11358f48, pr 4400). The file on sglang main is identical to the merge commit.
  • Both entries document the image lmsysorg/sglang:nightly-dev-20261021-c1c287a4 and the full python3 -m sglang.launch_server command for the recipe base: TP8/EP8, modelopt_fp4, flashinfer_trtllm MoE, trtllm_mha attention, fp8_e4m3 KV, page size 128, breakable prefill CUDA graph, DFLASH with DFlash2, 8 draft tokens with block verification, fa4 draft attention, mm-feature-transport cpu, muse parsers, tokenizer-worker-num 4, enable-http2 and watchdog-timeout 3600. The env block includes the new SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0, plus SGLANG_UNIFIED_RADIX_TREE_CORE_BACKEND=python, SGLANG_TIMEOUT_KEEP_ALIVE, OMP_NUM_THREADS, SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN, SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1 and PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True.
  • It documents every per-concurrency override in this PR line by line. TP8 c56: chunked prefill and max prefill 32768, interval 64, shortest-prefill-first, SWA 0.25, draft attention backend unset, layernorm SP + MegaMoE 8192, HiCache 160, SGLANG_TRTLLM_MHA_DECODE_SEQ_LEN_SPLITS=4, with no radix-core override. The new TP8 c64 has the same settings with admission and CUDA-graph batch 64 and interval 48. TP4 c24: batch 48, HiCache 180, mem-fraction-static 0.98, interval 48 and expandable_segments. The unchanged TP8 c1–c10 and TP4 c8–c16 overrides also match.
  • InferenceX-only settings excluded from upstream coverage: enable-metrics (scraped by the AIPerf harness); enable-cache-report (adds cached-token counts to response usage for the harness's cache accounting, with no change to scheduling, caching or kernels); PYTHONNOUSERSITE/PYTHONUNBUFFERED; HF_HUB_OFFLINE and the staged /model checkpoint and served-model-name plumbing; health-check polling; all benchmark.env client settings (TOKENIZER, AIPERF_*, AGENTIC_WARMUP_GRACE_PERIOD, CONC, KV_OFFLOADING); and the synthetic-acceptance env injected by the harness.
• No engine patches: the diff changes only YAML. The recipe has no setup scripts, patches, pip install, file overwrites or monkey-patching, so no waiver is needed.
• append-only: true: N/A. The appended changelog entry does not use append-only.
• Pareto coverage: there is one affected curve: Muse-Spark-Open · agentic-coding · b200_sglang · FP4. TP8 and TP4 points share this series, and no other config maps to it. The counts come from the 12 bmk_agentic_musespark_* artifacts of run 39756010759 (attempt 1, head 11358f48, image lmsysorg/sglang:nightly-dev-20261021-c1c287a4), with x = E2EL (s) and y = throughput.per_gpu.total_tput_tps, computed with infx.workflows.pareto_coverage:
  • P90 E2EL: 9 frontier points / 12 measured (TP8 c6, c8, c10, c56, c64; TP4 c8, c10, c16, c24).
  • P75 E2EL (reported separately): 6 frontier points / 12 measured (TP8 c10, c56, c64; TP4 c8, c16, c24).
  • No invalid or missing points.
• The PR branch currently conflicts with main only in the append-only inferencex-e2e/perf-changelog.yaml tail, which is resolved at merge. The new entry's pr-link uses the pull/XXX placeholder, which check-changelog accepted.

Signed: nvpohanh

[github-actions]

✅✅✅ Verdict: PASS ✅✅✅

Passed and not applicable checks

✅ Check 0 (CODEOWNER): PASS — @nvpohanh is a listed owner of inferencex-e2e/configs/nvidia-master.yaml and the **/b200-*/ srt-slurm recipe path; perf-changelog.yaml falls under the catch-all.

✅ Check 1 (Sweep on in-PR commit): PASS — the only PR commit 11358f48 has all 12 agentic / jobs and the agentic eval / job at success in run 39756010759; fixed-sequence lanes are skipped because the PR has no such configs.

✅ Check 2 (Evals): PASS — eval_results_all from run 39756010759 shows gsm8k em_strict 0.9757 (n=1319) on TP8 c64, above the 0.90 default threshold, using image lmsysorg/sglang:nightly-dev-20261021-c1c287a4, the same image as the PR config.

✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#46005 is MERGED (2026-10-23T08:04:02Z). muse-spark-open.jsx on sglang main (B200 #0 TP8·EP8 and #2 TP4·EP4, provenance 11358f48) documents the image, the base launch command, the env (including SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0 and the Python radix core), and every per-point override, including c56 (32768 chunk/prefill, interval 64), new c64 and TP4 c24 mem-fraction-static 0.98. The excluded enable-metrics/enable-cache-report and harness env are justified as InferenceX-only.

✅ Check 4 (Reuse command): PASS — /use 39756010759 posted on its own line by nvpohanh (COLLABORATOR); it names the Check 1 run.

✅ Check 5 (Latest checklist): PASS — the sign-off contains every item of the current template, including the recipe sub-item, all checked.

✅ Check 6 (Upstream image / engine order): PASS — framework: sglang on B200 uses upstream lmsysorg/sglang:nightly-dev-20261021-c1c287a4; this is an open-source engine entry, so the ordering rule does not apply.

✅ Check 7 (Deprecated models): PASS — musespark Agentic coding (SGLang DFlash2) is active in MODELS.md with no deprecated scenarios (review date 2026-10-23).

✅ Check 8 (No architecture hacks): PASS — the diff is YAML-only (image, env, scheduling, prefill, memory, HiCache and concurrency settings); there are no --hf-overrides, json-model-override-args or layer/expert skipping.

✅ Check 9 (Chat template): PASS — AgentX agentx profile replays multi-turn chat traces against /v1/chat/completions (agentx-standalone.md).

✅ Check 10 (No engine patches): PASS — no patches, pip install, file overwrites or monkey-patching; the PR changes only YAML.

✅ Check 11 (Golden AL): PASS — the harness injects SGLANG_SIMULATE_ACC_LEN=4.05, match-expected and real-draft-token (TP8 c64 srt-single-node-submission.json). DFlash speculative-num-draft-tokens: 8 maps to 7 speculative tokens, and musespark_dflash.yaml gives thinking_on/7 = 4.05.

➖ Check 12 (Append-only): N/A — the new changelog entry has no append-only: true.

✅ Check 13 (Draft precision): PASS — RadixArk/Muse-Spark-Open-DFlash2 ships as BF16 (dtype: bfloat16, no quantization_config). At sglang c1c287a4, a draft without quant metadata under modelopt_fp4 gets for_online_weight_quantization, which returns UnquantizedLinearMethod for LinearBase and quantizes only FusedMoE; dflash.py has no FusedMoE, so the draft serves in BF16. There is no draft quant, dtype or checkpoint override; draft KV follows the target's FP8 KV, which is consistent and allowed. SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not set in the recipe or the injected overrides, and its pinned default is EnvBool(False).

✅ Check 14 (Pareto coverage): PASS — the single curve musespark/agentic-coding/b200_sglang/fp4 (run 39756010759, attempt 1, 12 bmk_agentic_* artifacts, total_tput_tps vs E2EL) has 9/12 frontier points at P90 and 6/12 at P75 (infx.workflows.pareto_coverage), with no invalid points.

✅ Check 15 (PR description): PASS — the body's image, base env, per-point changes (c56 32768/interval 64/radix core, c64 batch 64/interval 48, TP4 c24 0.98) and concurrency lists match the code at 11358f48.

✅ Check 16 (NVIDIA overview): PASS — Muse-Spark-Open has a populated SGLang cell in the Framework Recipes table of the NVIDIA overview.

➖ Check 17 (Metrics publication): N/A — this is a direct SGLang deployment with no Dynamo TRT-LLM serving path.

Assessed commit: 11358f4843ec3e086014d3bba1a102e400944a59.

[Oseltamivir]

/use 39756010759

[github-actions]

@Oseltamivir staged run 39756010759.

• Muse-Spark-Open · agentic-traces · FP4 · b200_sglang · 2026-10-22

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow