Add nine Qwen3.5 NVFP4 GB300 Dynamo+SGLang AgentX points / 新增九个 Qwen3.5 NVFP4 GB300 Dynamo+SGLang AgentX 测试点

[nvpohanh]

[by Claude Code]

Summary

Adds nine Qwen3.5-397B-A17B NVFP4 GB300 Dynamo+SGLang AgentX points under two new config keys.

Config key	Recipe override	Topology	GPUs	Concurrency
qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg-src	override_3p1d_c704	3 × PP4 prefill, 1 × DEP8 decode	20	704
	override_1p1d_b10_c56	TP1 prefill + colocated TP2 decode, max-running-requests 10	3	56
	override_1p1d_b6_c38	TP1 prefill + colocated TP2 decode, max-running-requests 6	3	38
qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-src	override_tp2_c10	TP2, max-running-requests 1	2	5, 7, 10
	override_tp4_c7	TP4, max-running-requests 1	4	7
	override_tp8_c7	TP8 across two nodes, max-running-requests 1	8	7, 10

• Speculative decoding: MTP (NEXTN) with six draft steps, top-k 1 and seven draft tokens on prefill, decode and aggregated workers alike. The draft head is the checkpoint's shipped BF16 head: no speculative-draft-model-quantization and no SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE.
• Dynamo: the disaggregated points build Dynamo at b208989fbe, also the router version in the master config. It includes fix(sglang): count attention-DP ranks from attn_dp_size ai-dynamo/dynamo#16005, which registers every attention-DP rank of the DEP8 decode worker.
• Per-rank limits: the DEP8 decode worker caps 256 running requests (32 per DP rank) and captures decode CUDA graphs up to batch 32. The colocated TP2 decode workers capture graphs up to their request caps of 10 and 6.
• Cache directories: the recipes do not set SGLANG_CACHE_DIR, SGLANG_DG_CACHE_DIR or FLASHINFER_WORKSPACE_BASE, so SGLang, DeepGEMM and FlashInfer use their default cache locations. They do not share the /tmp paths of the existing Qwen3.5 GB300 recipes, which run a different image.
• Retired configs: removes the four existing Qwen3.5 FP4 GB300 Dynamo+SGLang AgentX configs, qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg, -agentic-disagg, -agentic-agg-pareto and -agentic-pp-pareto, with their recipes (agg-variants.yaml, disagg-variants.yaml, agg-pareto-variants.yaml, disagg-pp-pareto-variants.yaml). The nine new points supersede them on this curve, and no other config uses those recipes.

AI model disclosure

• Claude Opus 5.5 (claude-opus-5-5, via Claude Code):
  • prepared the recipes, master-config entries and changelog entry;
  • assembled this PR: removed the /tmp cache-directory overrides, ran the changelog validator, infx.matrix.changed and the srt-slurm generator for both keys, and wrote this description;
  • retired the four superseded configs and merged main.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

概要

在两个新的配置键下新增九个 Qwen3.5-397B-A17B NVFP4 GB300 Dynamo+SGLang AgentX 测试点。

测试点见上方表格：

• 分离式（...-agentic-disagg-src）：3 个 PP4 预填充 worker + 1 个 DEP8 解码 worker（20 GPU），并发 704；TP1 预填充 + 同节点 TP2 解码（3 GPU），并发 56 和 38，解码 max-running-requests 分别为 10 和 6。

• 聚合式（...-agentic-agg-src）：TP2 并发 5、7、10；TP4 并发 7；跨两个节点的 TP8 并发 7、10；max-running-requests 均为 1。

• 投机解码： 预填充、解码和聚合式 worker 统一使用 MTP（NEXTN），6 个草稿步、top-k 1、7 个草稿 token。草稿头为 checkpoint 自带的 BF16 头：不设置 speculative-draft-model-quantization，也不设置 SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE。

• Dynamo： 分离式测试点从 b208989fbe 构建 Dynamo，与 master 配置中的路由器版本一致。该版本包含 fix(sglang): count attention-DP ranks from attn_dp_size ai-dynamo/dynamo#16005，可为 DEP8 解码 worker 的每个注意力 DP rank 完成注册。

• 每个 rank 的上限： DEP8 解码 worker 的 max-running-requests 为 256（每个 DP rank 32），解码 CUDA graph 捕获到 batch 32。同节点 TP2 解码 worker 的 CUDA graph 分别捕获到其请求上限 10 和 6。

• 缓存目录： 配方不设置 SGLANG_CACHE_DIR、SGLANG_DG_CACHE_DIR 或 FLASHINFER_WORKSPACE_BASE，SGLang、DeepGEMM 和 FlashInfer 使用默认缓存位置，不与使用其他镜像的现有 Qwen3.5 GB300 配方共享 /tmp 路径。

• 退役配置： 移除现有的四个 Qwen3.5 FP4 GB300 Dynamo+SGLang AgentX 配置 qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg、-agentic-disagg、-agentic-agg-pareto 和 -agentic-pp-pareto，以及它们的配方（agg-variants.yaml、disagg-variants.yaml、agg-pareto-variants.yaml、disagg-pp-pareto-variants.yaml）。九个新测试点在该曲线上取代了它们，且没有其他配置使用这些配方。

AI 模型使用说明

• Claude Opus 5.5（claude-opus-5-5，通过 Claude Code）：
  • 编写配方、master 配置条目和 changelog 条目；
  • 整理本 PR：移除 /tmp 缓存目录设置，运行 changelog 校验、infx.matrix.changed 和两个配置键的 srt-slurm 生成器，并撰写本说明；
  • 退役四个已被取代的配置，并合并 main。

改动类型

配置变更。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

Code review found no issues

No high-confidence issues detected in this change.

This review covers commit b65b033, which is no longer the latest commit on this pull request; later commits are not covered by it.

[github-actions]

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=38860741275

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=38860741275

[nvpohanh]

[by Claude Code] I've started babysitting the pipeline of this PR. I'll keep monitoring it until it passes. If a job fails, I may rerun it when the cause is infrastructure, or push a fix or a merge of the latest main when the cause is in this PR.

[nvpohanh]

[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs (qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-src, qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg-src) will happen in sgl-project/sglang#44712, in the Agentic Long-Context Deployment section of the Qwen3.5 cookbook page.

This PR has no authorized /use <run_id> comment yet, so the cookbook shows the recipe at this PR's head (cd9176059) as "in progress". It switches to the measured run at the next cookbook sync after a run is pinned or this PR merges.

[nvpohanh]

cc @YAMY1234

[nvpohanh]

/use 38860741275

[github-actions]

@nvpohanh staged run 38860741275.

• Qwen-3.5-397B-A17B · agentic-traces · FP4 · gb300_dynamo-sglang · 2026-10-16

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[nvpohanh]

[by Claude Code] sgl-project/sglang#44712 has been merged. This PR's AgentX configs (qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg-src, qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg-src) are now in the Agentic Long-Context Deployment section of the SGLang Qwen3.5 cookbook: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.5#agentic-long-context-deployment

This PR's /use 38860741275 comment came after this cookbook round was synced, so the cookbook shows the recipe at cd9176059 as "in progress". That is the commit run 38860741275 measured, so the commands are already the measured ones; a later cookbook PR marks them verified.

[nvpohanh]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this.
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this.
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• Verified that every draft model and draft head is served as it ships: the draft that ships with the served checkpoint, at its stored precision, through the pinned upstream image's default handling, with the shipped and effective draft precision recorded in the additional detail section. No submission-side quantization, dtype override, checkpoint substitution, or patch may lower draft precision below that default, regardless of eval results or AL. Explicitly verified that SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE is not enabled in the effective recipe, including inherited settings; enabling it is prohibited going forward, and historical runs do not grant an exception. See Draft-model precision for what counts as the default and the MLPerf comparison.
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in infx/golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; target/verifier FLOPs at lower precisions is fine, given that the config passes private evals, but this does not permit lowering draft-model or draft-head precision below what ships. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• For affected Dynamo TRT-LLM benchmarks, verified that iteration/KV-cache metrics are collected and published under the pinned Dynamo implementation and effective engine settings. Resolve the installed Dynamo version separately from the container tag; use supported metrics controls without requiring KV events. Record the version and configuration evidence in the additional detail section. See verifier Check 17. N/A for unaffected serving paths or frameworks without these Dynamo settings.
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every affected vLLM/SGLang production serving configuration in this PR, including Dynamo deployments, is documented in the official vLLM recipes and/or the SGLang cookbook, for both single-node and multi-node aggregated and disaggregated serving. The documentation covers production-useful flags, environment variables, and launch commands (including prefill, decode, and router/frontend commands for disaggregation); InferenceX-only benchmark/harness settings are excluded with reasoning in the additional detail section:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang cookbook source repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.
• Reported measured throughput/E2EL Pareto counts and evidence per affected curve (≥5 points strongly recommended). Below 5 or unverifiable: tag a core maintainer for review; recorded admin bypass required before merge. N/A if no curves are affected. Details.

Additional detail section:

• Assessed head: cd91760593aee18ce10a9331e68415e815236eba. Checklist template matches main at fce913b9e. The diff is config-only: two new recipe bundles, two new and four retired nvidia-master.yaml entries, and one appended perf-changelog.yaml entry. It adds no comments, and the changelog validator, infx.matrix.changed and the srt-slurm generator pass for both new keys.
• PR validation: Run Sweep 38860741275 (pull_request, head cd91760593, full-sweep-fail-fast) completed success. All nine multi-node agentic / benchmark jobs (canary TP2 c5 included) and all three multi-node agentic eval / jobs succeeded; none of them were skipped. No benchmark point dropped a request to an error (only warmup records are excluded). Pinned with /use 38860741275.
• Evals: eval_results_all of run 38860741275, gsm8k em_strict 0.9765 (1P1D c38), 0.9765 (1P1D c56) and 0.9788 (3P1D c704). All are above the qwen3.5 gsm8k threshold of 0.94 in infx/evals/thresholds.yaml, on the same image as the configs (lmsysorg/sglang:dev-qwen35-agentx-gb300-ccebaceeeca3).
• Chat template: the AgentX client (agentx profile) drives the OpenAI chat endpoint. The run's profile_export_aiperf.json records input_config.endpoint.type: chat, streaming: true.
• Draft precision (shipped = effective = BF16):
  • Shipped: the embedded MTP head of nvidia/Qwen3.5-397B-A17B-NVFP4 (HF 12b28061). hf_quant_config.json and config.json exclude mtp.layers.0* from NVFP4, and the safetensors index lists 1553 mtp.* tensors with no scale tensors, so the head is stored in BF16. On the GB300 cluster, runners.yaml maps this model to the staged Qwen3.5-397B-A17B-NVFP4-V2 copy.
  • Pinned image default: _mtp_quant_config in python/sglang/srt/models/qwen3_5_mtp.py at SGLang ccebaceeeca3 builds the MTP module unquantized when the modelopt_mixed config lists no mtp.* quantized layer.
  • Effective: the recipes set no speculative-draft-model-quantization, draft path, draft dtype or draft KV override. The Qwen3_5ForCausalLMMTP load logs show 16.04 GB (TP1 prefill), 8.07 GB/rank (TP2 decode) and 4.02 GB/rank (TP4 agg), about 16 GB in total. That matches a BF16 MTP layer (~13.2 GB) plus its embedding, not NVFP4 (~3.7 GB).
  • Draft MoE kernels: speculative-moe-runner-backend: flashinfer_cutedsl has no unquantized path in UnquantizedFusedMoEMethod.create_moe_runner, so the draft falls back to the BF16 Triton runner. SGLANG_Q35MF_DRAFT_MOE_GEMV (SingleTokenDraftMoe) requires UnquantizedFusedMoEMethod and BF16 operands. With SGLANG_MOE_NVFP4_DISPATCH=1 on the DEP8 decode, FlashInfer A2A uses NVFP4 only for layers with an input_global_scale (_effective_dispatch_type), so the BF16 draft dispatches in BF16.
  • SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE: absent from both recipes (base and overrides), from the rendered srt recipes, and from the worker env fingerprints of the run.
• Golden AL: infx/golden_al_distribution/qwen3.5_mtp.yaml, qwen3.5-397b-a17b-nvfp4, thinking_on, 6 speculative tokens: 4.80. The recipes use speculative-num-steps: 6, speculative-eagle-topk: 1 and speculative-num-draft-tokens: 7. The job log shows THINKING_MODE: thinking_on, and the prefill, decode and agg worker env fingerprints show SGLANG_SIMULATE_ACC_LEN=4.8 and SGLANG_SIMULATE_ACC_METHOD=match-expected.
• MODELS.md: Qwen3.5-397B-A17B agentic coding at fp4 is active.
• No architecture hacks: there is no --hf-overrides, --json-model-override-args or model-file edit. The SGLANG_Q35MF_* switches select kernels and scheduling paths for the same computation.
• Dynamo TRT-LLM metrics: N/A. The PR affects only dynamo-sglang configs.
• Upstream image: lmsysorg/sglang:dev-qwen35-agentx-gb300-ccebaceeeca3 is published under the lmsysorg Docker Hub org (amd64 and arm64).
• Engine-first ordering: the PR adds only dynamo-sglang (open-source engine) entries.
• SGLang cookbook: sgl-project/sglang#44712 is MERGED (2026-10-16T11:31:09Z). The published page is Qwen3.5 — Agentic Long-Context Deployment.
  • Coverage: its data (docs/src/snippets/agentx/Qwen/qwen3.5.jsx, provenance cd91760593) covers both keys and all six deployments: agg TP2, TP4 and TP8, and disagg 3P1D c704, 1P1D c56 and 1P1D c38.
  • Comparison method: I decoded the published commands with the page's own decoder and compared them role by role with the effective recipes at cd91760593. Every production-useful worker flag and environment variable matches, including the SGLANG_Q35MF_* switches, HiCache, MTP and parallelism.
  • Frontend and Dynamo: the Dynamo frontend args match (router-mode kv, session-affinity TTL 3600, router-replica-sync, router-temperature 1.0 and the disabled thresholds). c704 shows two frontends behind nginx session affinity, as the recipe runs them. The 1P1D jobs ran a single frontend. The Dynamo source commit b208989fbe is given as the install step.
  • Excluded as InferenceX-only:
    • SGLANG_SIMULATE_ACC_*: synthetic acceptance.
    • --enable-metrics, --enable-cache-report, --decode-log-interval, --log-level, MC_TE_METRIC, NO_COLOR, PYTHONUNBUFFERED: metrics and logging, with no serving effect.
    • PIP_BREAK_SYSTEM_PACKAGES: lets srt-slurm install Dynamo at job start; covered by the install step.
    • --nccl-port: collision avoidance for colocated workers; SGLang picks a free port.
    • --model-path /model/ and --disaggregation-bootstrap-port 31000: placement values, normalized to the HF ID and the default 8998.
    • AIPERF_* benchmark env and Slurm, health-check and sbatch/srun settings.
• No patches: there are no patches, setup scripts, source mounts or site-packages edits, and the SGLang image runs as shipped. The disagg recipe installs upstream ai-dynamo/dynamo at b208989fbe, which contains ai-dynamo/dynamo#16005. It uses srt-slurm's standard dynamo.install, as other merged Dynamo+SGLang recipes do. No waiver is needed.
• append-only: N/A. The changelog entry does not use append-only: true.
• Pareto coverage: the PR affects one curve, Qwen3.5 / agentic-coding / GB300 dynamo-sglang / FP4 (run 38860741275, p90_e2el vs tput_per_gpu). infx.workflows.pareto_coverage (helper identical to main) reports 9 measured points, 0 invalid, and 7 frontier points (PASS):
  • agg TP8 c7 (6.30 s, 5,369)
  • agg TP4 c7 (7.44 s, 10,381)
  • agg TP2 c5 (8.70 s, 16,288)
  • agg TP2 c7 (10.45 s, 19,885)
  • 1P1D c38 (18.11 s, 56,209)
  • 1P1D c56 (22.34 s, 70,209)
  • 3P1D c704 (24.64 s, 138,945)

Signed: nvpohanh

[github-actions]

✅✅✅ Verdict: PASS ✅✅✅

Passed and not applicable checks

✅ Check 0 (CODEOWNER): PASS — @nvpohanh is listed as an owner of inferencex-e2e/configs/nvidia-master.yaml and of /inferencex-e2e/benchmarks/*_node/srt-slurm-recipes/**/gb300-*/, which covers every changed recipe; perf-changelog.yaml falls under the catch-all owner.

✅ Check 1 (Sweep on in-PR commit): PASS — Head cd91760593 is in the PR. In run 38860741275 (pull_request, attempt 1), all 8 multi-node agentic / jobs, the TP2 c5 canary and all 3 multi-node agentic eval / jobs finished success, none skipped; the generic single-node/eval / jobs were skipped only because the PR changes no single-node configs.

✅ Check 2 (Evals): PASS — eval_results_all reports gsm8k em_strict of 0.9765 (1P1D c38), 0.9765 (1P1D c56) and 0.9788 (3P1D c704), with n=1319 each and infrastructure_success true; all three ran on lmsysorg/sglang:dev-qwen35-agentx-gb300-ccebaceeeca3, the image in both config entries.

✅ Check 3 (Upstream recipe): PASS — sgl-project/sglang#44712 is MERGED (2026-10-16T11:31:09Z). Its docs/src/snippets/agentx/Qwen/qwen3.5.jsx on main (provenance cd91760593) covers all six deployments: agg TP2, TP4 and TP8 (2 nodes) with the sglang-router command, and disagg 1P1D b6/b10 and 3P1D PP4/DEP8 with Dynamo frontend, prefill and decode commands, the Dynamo b208989fbe install and nginx for c704. Role-by-role, the worker args, SGLANG_Q35MF_*/HiCache/MTP env and parallelism match the pinned recipes. The excluded settings (synthetic acceptance, metrics/logging, nccl-port, bootstrap port, model path, AIPERF) are InferenceX-only and are justified in the sign-off.

✅ Check 4 (Reuse command): PASS — nvpohanh (COLLABORATOR) posted /use 38860741275 on its own line, which is the passing run on the head commit.

✅ Check 5 (Checklist template): PASS — The sign-off contains every item of the current PR_REVIEW_CHECKLIST.md template, including the merged-upstream sub-item and the Pareto item, and every item is checked.

✅ Check 6 (Upstream image / ordering): PASS — Both new entries use lmsysorg/sglang:dev-qwen35-agentx-gb300-ccebaceeeca3, which is in the upstream lmsysorg org. It is built from sgl-project/sglang branch nvidia/qwen35-agentx-gb300, not main. Both entries are dynamo-sglang (open-source engine), so the ordering rule does not apply.

✅ Check 7 (Deprecated models): PASS — MODELS.md lists qwen3.5 Agentic coding at fp8/fp4 as active on 2026-10-16. Both new configs are fp4 agentic-coding.

✅ Check 8 (Architecture hacks): PASS — There is no --hf-overrides, --json-model-override-args or model-file edit. At the pinned SGLang, SGLANG_Q35MF_ATTN_GATE_QUANT/GDN_NORM_FP8 only fuse the existing static FP8 input quantization of ModelOptFp8LinearMethod projections, and the other Q35MF_* switches are kernel and scheduling paths for the same computation.

✅ Check 9 (Chat template): PASS — The run's profile_export_aiperf.json (TP2 c5) records endpoint type: chat for the AgentX client.

✅ Check 10 (Engine patches): PASS — The PR adds no patch files, setup scripts, inline source edits or engine wheels, so the SGLang image runs as shipped. The disagg recipe uses srt-slurm's standard dynamo.install of upstream ai-dynamo/dynamo@b208989fbe, which contains #16005 (e18e53fa3c).

✅ Check 11 (Agentic golden AL): PASS — The synthetic acceptance is injected by infx/srt_slurm/synthetic_acceptance.py, not hard-coded in the recipes. The run logs show SGLANG_SIMULATE_ACC_LEN=4.8, match-expected and real-draft-token on the prefill, decode and agg workers (c38, c704, TP4) with THINKING_MODE: thinking_on, matching the golden qwen3.5_mtp.yaml value for thinking_on with 6 speculative tokens (4.80) at speculative-num-steps: 6 / num-draft-tokens: 7.

➖ Check 12 (Append-only): N/A — The new changelog entry does not set append-only: true.

✅ Check 13 (Draft precision): PASS — The NVFP4 checkpoint's hf_quant_config.json excludes mtp.layers.0*, so the shipped MTP head is BF16. _mtp_quant_config in qwen3_5_mtp.py at ccebaceeeca3 builds the MTP module unquantized for serialized NVFP4/mixed checkpoints. Neither recipe sets a draft quantization, draft path or draft dtype. SGLANG_NVFP4_CKPT_FP8_NEXTN_MOE defaults to EnvBool(False) in the pinned environ.py, is removed by this diff and is absent from the run's worker env fingerprints.

✅ Check 14 (Pareto coverage): PASS — There is one curve, Qwen3.5 / agentic-coding / GB300 dynamo-sglang / fp4 / p90_e2el vs tput_per_gpu, from results_bmk of run 38860741275 (attempt 1, cd91760593, a single image). pareto_coverage reports 9 measured points, 0 invalid and 7 frontier points; TP8 c10 and TP2 c10 are dominated.

✅ Check 15 (PR description): PASS — The body's claims all match the pinned recipes: the nine points and topologies, decode caps 256/10/6 and CUDA-graph sizes 32/10/6, the absence of cache-dir overrides, Dynamo b208989fbe containing #16005, and the four retired configs (no other config uses their recipe dir).

✅ Check 16 (NVIDIA overview): PASS — NVIDIA's recipes overview Framework Recipes table has a populated SGLang cell for Qwen3.5, which links to the SGLang Qwen3.5 cookbook.

➖ Check 17 (Metrics publication): N/A — The PR affects only dynamo-sglang configs, which do not use the Dynamo TRT-LLM metrics settings.

Assessed commit: cd91760593aee18ce10a9331e68415e815236eba.

[functionstackx]

added some comments on slack about the weridness of the e2el

[nvpohanh]

[by Claude Code] The SGLang cookbook update of this PR's AgentX configs (qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg, qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg) will happen in sgl-project/sglang#45111, in the Agentic Long-Context Deployment section of the Qwen3.5 cookbook page. The configs have been live in that section since sgl-project/sglang#44712; sgl-project/sglang#45111 refreshes them with the pin below.

This PR's /use comment pins run 38860741275, which measured cd9176059. That commit does not have these config keys (the PR added or renamed them since), so the cookbook shows the recipe at this PR's head (6d447b6ae) as "in progress". A later cookbook PR marks it measured once a run of these keys is pinned or this PR merges.

[nvpohanh]

[by Claude Code] sgl-project/sglang#45111 has been merged. This PR's AgentX configs (qwen3.5-fp4-gb300-dynamo-sglang-agentic-agg, qwen3.5-fp4-gb300-dynamo-sglang-agentic-disagg) are now in the Agentic Long-Context Deployment section of the SGLang Qwen3.5 cookbook: https://docs.sglang.io/cookbook/autoregressive/Qwen/Qwen3.5#agentic-long-context-deployment

This PR's /use comment pins run 38860741275, which measured cd9176059. That commit does not have these config keys (the PR added or renamed them since), so the cookbook shows the recipe at this PR's head (6d447b6ae) as "in progress". A later cookbook PR marks it measured once a run of these keys is pinned or this PR merges.