Repository navigation
[AgentX] MiniMax-M3.1-Flash B200/GB200/B300 SGLang: add conc 1 low-latency configs / [AgentX] MiniMax-M3.1-Flash B200/GB200/B300 SGLang:新增 conc 1 低延迟配置 - #4509
Conversation
…ang AgentX Adds the MSA-V1 MonoKernel TP4 point at conc 1, following the GB300 recipe. 为 MiniMax-M3.1-Flash 的 B200、GB200、B300 SGLang AgentX 新增 conc 1 低延迟配置,沿用 GB300 的 MSA-V1 MonoKernel TP4 设置。
|
Thanks for the contribution!
中文感谢你的贡献!
|
| # Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1 | ||
| # DSpark steps; larger batches take the standard path. |
There was a problem hiding this comment.
🟡 nit (optional): the new recipes add a comment saying where the config came from, which AGENTS.md's "No history" rule bans. sweep:# Follows the GB300 Low-Latency cell The comment above args: says what the recipe was copied from. The rule says derivation belongs in git and perf-changelog. Fix: drop "Follows the GB300 Low-Latency cell:" from each of the three new recipes. Keep only the technical part, for example # MSA-V1 MonoKernel serves batch-size-1 DSpark steps; larger batches take the standard path.
Why this was flagged
The three new recipe files b200-fp4-mtp/agentic-low-latency.yaml:24, b300-fp4-mtp/agentic-low-latency.yaml:24 and gb200-fp4-mtp/agentic-low-latency.yaml:24 each add # Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1. AGENTS.md "Code comments (mandatory)" covers recipes and says: "Do not write comments that describe how the code evolved, where it was derived from, what it replaced". The opening clause describes where the recipe was derived from, not why a value was chosen. Behaviour does not change. The cost is a convention breach that leaves maintainers a stale pointer once the GB300 recipe changes.
Verification: Each new recipe (b200-fp4-mtp, b300-fp4-mtp and gb200-fp4-mtp agentic-low-latency.yaml) adds the same two lines at 24-25, just above args:, starting "# Follows the GB300 Low-Latency cell". Line 65 of AGENTS.md says: "No history. Do not write comments that describe... where it was derived from". That clause is exactly what that rule bans. No runtime behavior is affected.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40535321029 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40535321029 |
Keeps only the technical note, per the no-history comment rule. 只保留技术说明,符合不写来源历史的注释规则。
|
/use 40194550246 |
|
@ispobock staged run 40194550246.
This run remains available across future |
…g-agentx-low-latency
|
As a PR reviewer and CODEOWNER, I have reviewed this and have Ignore missing checklist, do the other checks |
❌❌❌ REJECTED ❌❌❌❗❗ WARNING ❗❗ NVIDIA official recipes overview: model/framework coverage is missing or unverified. See Check 16. @Oseltamivir The sign-off has no checklist and no additional detail section. It doesn't link a recipe, it doesn't give draft-precision evidence, and it doesn't show from server logs that the MSA MonoKernel path ran during the evals. The sweep itself is green on an in-PR commit (run 40194550246 at ❌ Check 2 (Evals pass): FAIL — No eval is proven to have run the changed path (2c). The gsm8k accuracy is in line with peers: 0.9666 B200, 0.9620 GB200, 0.9651 B300, all ❌ Check 3 (Upstream recipe): FAIL — The sign-off links no merged or published recipe. The SGLang cookbook MiniMax-M3.1 page lists its Low-Latency/MonoKernel cell for B300/GB300 and B200 only, not GB200. Link the page, show that it covers TP4, DSPARK block 7 / 8 draft tokens, ❌ Check 5 (Latest checklist): FAIL — The sign-off contains no items from the current PR_REVIEW_CHECKLIST.md. Every template item is missing or unchecked, and "Ignore missing checklist" is not an explanation.
Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — The changed paths are owned by NVIDIA reviewers plus ✅ Check 1 (In-PR sweep): PASS — ✅ Check 4 (Reuse command): PASS — ispobock (COLLABORATOR) posted ✅ Check 6 (Upstream image): PASS — All 3 entries are ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (Architecture hacks): PASS — No ✅ Check 9 (Chat template): PASS — AgentX throughput uses golden simulated acceptance (Check 11). Evals drive ✅ Check 10 (Engine patches): PASS — The new recipes have no setup scripts, patches or source edits, and the launch path is the same one the sibling SGLang AgentX recipes use. ✅ Check 11 (Golden AL): PASS — The connector injects ➖ Check 12 (Append-only): N/A — The new changelog entry has no ➖ Check 17 (Metrics publication): N/A — These are direct SGLang configs with no Dynamo; ✅ Check 18 (Recipe/image/sweep consistency): PASS — Each config is TP4, 4 GPUs, conc 1 in both master and recipe, and all keys generate. The image digest is already in use and pinned. No points were removed. The changelog description is accurate, and the Assessed commit: |
# Conflicts: # inferencex-e2e/perf-changelog.yaml
|
MonoKernel dispatch evidence for the evals in run 40194550246, from B300 ( B200 ( GB200 ( During the evals every TP0 |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have Ignore missing checklist, do the other checks, look at PR comments & sgl-project/sglang#46506 |
❌❌❌ REJECTED ❌❌❌❗❗ WARNING ❗❗ NVIDIA official recipes overview: model/framework coverage is missing or unverified. See Check 16. @Oseltamivir Two things block this: the sign-off still has no checklist, and the linked sgl-project/sglang#46506 (merged 2026-10-26) publishes the MonoKernel Low-Latency recipe only for B300 and GB300, so B200 and GB200 have no upstream coverage. The sweep is green on in-PR commit ❌ Check 2 (Evals pass): FAIL — This verifier could not confirm that the changed path ran (2c). The MonoKernel dispatch lines in ispobock's comment come from the ❌ Check 3 (Upstream recipe): FAIL — B200 and GB200 Low-Latency (TP4 MonoKernel) have no merged or published recipe. B300 matches exactly: the published cookbook ❌ Check 5 (Latest checklist): FAIL — The sign-off contains none of the items in the current PR_REVIEW_CHECKLIST.md. "Ignore missing checklist" is not an explanation for leaving every item out.
Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — The NVIDIA master-config and recipe paths are owned by the NVIDIA reviewers plus ✅ Check 1 (In-PR sweep): PASS — ✅ Check 4 (Reuse command): PASS — ispobock (COLLABORATOR) posted ✅ Check 6 (Upstream image): PASS — All 3 entries are ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (Architecture hacks): PASS — There are no ✅ Check 9 (Chat template): PASS — AgentX replays chat traces with golden simulated acceptance (Check 11), and the evals drive chat completions. ✅ Check 10 (Engine patches): PASS — The new recipes have no setup scripts, patches or source edits. No ✅ Check 11 (Golden AL): PASS — All 3 agentic job logs show ➖ Check 12 (Append-only): N/A — The new changelog entry has no ➖ Check 17 (Metrics publication): N/A — These are direct SGLang configs with no Dynamo, and ✅ Check 18 (Recipe/image/sweep consistency): PASS — Each config is TP4 on 4 GPUs at conc 1 in both master and recipe, and all keys generate at the pinned SHA. The image digest is already in use, so nothing newly referenced needs resolving. No points were removed, and the changelog description is accurate with pr-link set to 4509. Assessed commit: |
…g-agentx-low-latency # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
[by Claude Code] The SGLang cookbook integration of this PR's AgentX configs ( This PR's |
|
[by Claude Code] sgl-project/sglang#46807 has been merged. This PR's AgentX configs ( This PR's |
…g-agentx-low-latency # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX configs ( This PR's |
|
[by Claude Code] sgl-project/sglang#46948 has been merged. This PR's AgentX configs ( This PR's |
|
/use 40535321029 |
|
@Oseltamivir staged run 40535321029.
This run remains available across future |
|
/use 40535321029 |
…g-agentx-low-latency # Conflicts: # inferencex-e2e/perf-changelog.yaml
|
@ispobock staged run 40535321029.
This run remains available across future |
|
As a PR reviewer and CODEOWNER, I have reviewed this and have Ignore missing checklist, do the other checks, look at PR comments for recipe link |
❌❌❌ REJECTED ❌❌❌@Oseltamivir The sign-off still has no checklist, and its body only points to the PR comments instead of linking the recipe. Neither can be waived here. The sweep and evals passed on in-PR commit ❌ Check 2 (Evals pass): FAIL — Could not verify that the changed path ran (2c). The MonoKernel dispatch lines that ispobock cites are inside the ❌ Check 3 (Upstream recipe): FAIL — The sign-off itself links no merged or published recipe; "look at PR comments for recipe link" does not meet (a). Coverage does exist: sgl-project/sglang#46948 (merged 2026-10-28) puts B200, GB200 and B300 ❌ Check 5 (Latest checklist): FAIL — The sign-off contains none of the items in the current PR_REVIEW_CHECKLIST.md. "Ignore missing checklist" does not explain why the items are missing.
Passed and not applicable checks✅ Check 0 (CODEOWNER): PASS — Oseltamivir is listed in CODEOWNERS. The changed paths are owned by the NVIDIA owners plus ✅ Check 1 (Sweep on in-PR commit): PASS — ✅ Check 4 (Reuse command): PASS — ✅ Check 6 (Upstream image): PASS — ✅ Check 7 (Deprecated models): PASS — ✅ Check 8 (No architecture hacks): PASS — No ✅ Check 9 (Chat template): PASS — These are AgentX trace-replay points whose acceptance is simulated at the golden AL (Check 11), not taken from raw-completion traffic. ✅ Check 10 (No engine patches): PASS — The new recipes have no setup scripts, patches, or source edits, and the launcher path is unchanged for these configs. ✅ Check 11 (Golden AL): PASS — The launch log applies ✅ Check 12 (Append-only): PASS — Each key gains exactly one TP4 c1 point (B200 and GB200 12→13, B300 9→10) from the new ✅ Check 15 (PR description): PASS — The body's claims (conc 1, TP4, MSA MonoKernel at batch size 1, added to the existing keys, separate recipe file, follows the GB300 recipe from #4456) match the pinned diff. ✅ Check 16 (NVIDIA overview): PASS — The NVIDIA overview lists MiniMax-M3.1-Flash with SGLang, which covers B200, GB200 and B300. ➖ Check 17 (Metrics publication): N/A — These are direct SGLang configs with no Dynamo worker settings. ✅ Check 18 (Recipe consistency): PASS — Recipes resolve and have no generation errors. Master TP4/c1/kv none matches the recipe's 4 GPUs, Assessed commit: |
|
[by Claude Code] The SGLang cookbook update of this PR's AgentX configs ( This PR's |
|
[by Claude Code] sgl-project/sglang#47027 has been merged. This PR's AgentX configs ( This PR's |
|
Ingest report — ✅ Results ingested into the Ingest workflow · Merge Ingest run · Benchmark source run
benchmarkFiles 3, rawBenchmarkRows 3, coveredByPerJobFiles 0, purgedPoints 0, failedRunRows 0, runStatsNew 0, runStatsDuplicate 0, evalSamplesNew 0, gpuMetricSamplesNew 65700, powerPublicationPoints 3, changelogEntriesWritten 1 Counts cover only this run. Rows: benchmark_results, eval_results, agentic rows linked to a trace, gpu_metrics series. This comment updates if the ingest is rerun. 中文Ingest 报告 — ✅ 结果已写入 上表仅统计本次运行写入的数据,各行单位依次为 benchmark_results 行、eval_results 行、已关联 trace 的 agentic 行和 gpu_metrics 序列。重新运行 ingest 时本评论会原地更新。 |
Add a Low-Latency point at conc 1 for the B200, GB200 and B300 SGLang AgentX configs, following the GB300 recipe from #4456: TP4 with the MSA-V1 MegaKernel serving batch-size-1 DSpark steps. The point is added to each SKU's existing config, and the recipe lives in its own file, so existing points are unchanged.
中文
为 B200、GB200、B300 的 SGLang AgentX 配置新增 conc 1 低延迟点,沿用 #4456 中 GB300 的设置:TP4,由 MSA-V1 MegaKernel 处理 batch size 1 的 DSpark 步骤。该点加在各 SKU 现有的配置中,recipe 单独成文件,现有的点不受影响。
AI model disclosure
Prepared with Claude Code using
claude-opus-5-5. Reviewed by the human author before submission.