Skip to content

Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated EAGLE Dynamo-vLLM / 刷新 MiniMax-M3 NVFP4 B300 8k1k 分离式 EAGLE Dynamo-vLLM 配置 - #2314

Merged
Oseltamivir merged 5 commits into
mainfrom
agent/refresh-minimax-m3-b300-eagle
Jul 26, 2026
Merged

Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated EAGLE Dynamo-vLLM / 刷新 MiniMax-M3 NVFP4 B300 8k1k 分离式 EAGLE Dynamo-vLLM 配置#2314
Oseltamivir merged 5 commits into
mainfrom
agent/refresh-minimax-m3-b300-eagle

Conversation

@jasonlizhengjian

@jasonlizhengjian jasonlizhengjian commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Refresh five MiniMax-M3 NVFP4 B300 8k1k disaggregated Dynamo-vLLM EAGLE3 recipes to vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9.
  • Remove the deprecated VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm override from those five refreshed recipes, matching the image-compatibility change established in Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 #2120.
  • Retain the 2P1D DEP4 recipe on its legacy image and all-reduce settings under a separate matrix key after the refreshed image showed a correctness regression.
  • Preserve all six EAGLE3 topologies, speculative-token settings, chat template, random range ratio, and 12 total concurrency points.
  • This PR contains no B200 or non-EAGLE recipe changes.

Validation

  • python -m pytest utils/matrix_logic/ utils/changelog_gate_tests/ -q: 300 passed.
  • Generated scope: five refreshed jobs with 10 concurrency points plus one legacy DEP4 job with two points.

中文说明

  • 将五个 MiniMax-M3 NVFP4 B300 8k1k 分离式 Dynamo-vLLM EAGLE3 配方更新到 vllm/vllm-openai:nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9
  • 从这五个更新后的配方中移除已弃用的 VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm 覆盖配置,与 Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 #2120 的镜像兼容性调整保持一致。
  • 由于新镜像出现正确性回退,将 2P1D DEP4 配方保留在旧镜像和原有 all-reduce 设置下,并拆分为独立 matrix key。
  • 保留全部六种 EAGLE3 拓扑、推测 token 设置、chat template、随机范围比例和共 12 个并发点。
  • 本 PR 不包含 B200 或非 EAGLE 配方变更。

验证

  • python -m pytest utils/matrix_logic/ utils/changelog_gate_tests/ -q:300 项测试通过。
  • 生成范围:五个更新任务共 10 个并发点,另加一个旧版 DEP4 任务的两个并发点。

Bump the six existing 8k1k Dynamo-vLLM EAGLE recipes to the refreshed upstream vLLM nightly and remove the deprecated TRT-LLM FlashInfer all-reduce override.

中文:将现有六个 8k1k Dynamo-vLLM EAGLE 配方更新到新版上游 vLLM nightly 镜像,并移除已弃用的 TRT-LLM FlashInfer all-reduce 覆盖配置。
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs


感谢你的贡献!请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人,你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板,包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have

如需进行 PR 验证,请为此 PR 添加 full-sweep-fail-fast 标签(强烈推荐)— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动(flake),重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

Append the MiniMax-M3 NVFP4 B300 disaggregated EAGLE config key for PR #2314.

中文:为 PR #2314 追加 MiniMax-M3 NVFP4 B300 分离式 EAGLE 配置键,以触发对应扫描。
@jasonlizhengjian jasonlizhengjian added the non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim) label Jul 23, 2026
@jasonlizhengjian
jasonlizhengjian marked this pull request as ready for review July 23, 2026 17:36
@jasonlizhengjian
jasonlizhengjian requested a review from a team July 23, 2026 17:36
@claude

claude Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Claude finished @jasonlizhengjian's task in 2m 15s —— View job


Review

LGTM - no blocking issues found

Verified: all six recipe YAMLs and the minimaxm3-fp4-b300-dynamo-vllm-mtp master-config key point to the same new tag with no stale references to the old one; the perf-changelog.yaml entry is correctly appended at the end of the file with the matching config key; the VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm removal matches the same removal in #2120 (note that PR is still open, not merged); topologies, EAGLE3/speculative settings, and the 12 concurrency points are untouched, matching the master config's six CONFIG_FILE references.

Two things I could not verify from this sandbox (network/exec restrictions): that the nightly-4080263bb2c5d10deac17aaeb88e0823bc35bca9 tag actually exists on Docker Hub (a prior changelog entry notes nightlies can be garbage-collected, so worth a quick docker manifest inspect before the sweep), and the local pytest run — the CI test jobs cover the latter.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — mechanical image-tag refresh and deprecated env-var removal across the six MiniMax-M3 B300 EAGLE recipes.

What was reviewed:

  • Confirmed all six recipe diffs are identical mechanical changes (image tag bump + drop VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm from prefill/decode env), no topology, speculative-config, or concurrency changes.
  • Verified configs/nvidia-master.yaml image reference and perf-changelog.yaml entry match the recipe changes.
  • Cross-checked against the established precedent (#2120) for the same env-var removal pattern.
Extended reasoning...

Overview

Refreshes the vLLM nightly container image tag and removes a deprecated VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm env override across six existing MiniMax-M3 NVFP4 B300 8k1k disaggregated EAGLE3 recipe YAMLs, plus the corresponding configs/nvidia-master.yaml image pin and a perf-changelog.yaml entry. No topology, speculative-decoding, chat-template, or concurrency settings are touched.

Security risks

None. This is a benchmark-recipe config change (container tag bump, env var removal) with no code execution paths, auth, or data-handling logic involved.

Level of scrutiny

Low. All six file diffs are byte-for-byte identical in shape (only the file name/topology differs), following an established precedent (#2120) for dropping the same deprecated env var. I independently verified the current main branch already reflects the identical image tag and env-var removal (landed via a separate direct commit, d36d9b3), and that a follow-up commit (8985f5f) explicitly registers this PR number in the changelog to drive its validation sweep — this matches the repo'''s established workflow for this kind of recipe refresh rather than indicating drift or a stale/duplicate PR.

Other factors

The PR description reports 300 passing tests for the matrix-logic/changelog-gate test suites, and the change is scoped narrowly (no B200 or non-EAGLE recipes touched, per the author'''s own description). No outstanding review comments need addressing.

@github-actions

Copy link
Copy Markdown
Contributor

2 similar comments
@github-actions

Copy link
Copy Markdown
Contributor

@github-actions

Copy link
Copy Markdown
Contributor

Signed-off-by: Jason Li <lijas@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Signed-off-by: Jason Li <lijas@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

1 similar comment
@github-actions

Copy link
Copy Markdown
Contributor

@xinli-sw

Copy link
Copy Markdown
Collaborator

/reuse-sweep-run

@Ankur-singh Ankur-singh changed the title [WIP] Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated EAGLE Dynamo-vLLM / 刷新 MiniMax-M3 NVFP4 B300 8k1k 分离式 EAGLE Dynamo-vLLM 配置 Refresh MiniMax-M3 NVFP4 B300 8k1k disaggregated EAGLE Dynamo-vLLM / 刷新 MiniMax-M3 NVFP4 B300 8k1k 分离式 EAGLE Dynamo-vLLM 配置 Jul 23, 2026
@Ankur-singh

Copy link
Copy Markdown
Collaborator

As a PR reviewer and CODEOWNER, I have reviewed this and have:

  • Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
  • Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
  • Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. — https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30042427571
  • Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. — https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30042427571
  • Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
  • For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
  • Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
  • If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
  • If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
  • Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
    • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
  • Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
  • If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

  • Scope: EAGLE3 DISAGG multi-node Dynamo-vLLM refresh for MiniMax-M3 NVFP4 B300 8k1k. Two config keys: minimaxm3-fp4-b300-dynamo-vllm-mtp (5 EAGLE3 topologies refreshed to vllm/vllm-openai:nightly-4080263bb2…, upstream repo; drops the deprecated VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm override from prefill+decode, matching PR Update vLLM MiniMax-M3 image tags / 更新 vLLM MiniMax-M3 镜像标签 #2120) and the new minimaxm3-fp4-b300-dynamo-vllm-mtp-legacy-dep4 (the DEP4 topology, conc 256/512, split out and intentionally retained on the legacy image vllm/vllm-openai:nightly-8e981630c9…). All 12 concurrency points across 6 topologies preserved.
  • Validation & evals: Run Sweep https://github.com/SemiAnalysisAI/InferenceX/actions/runs/30042427571 (head b1c771c5) is green top-level with non-skipped multi-node 8k1k benchmark jobs and multi-node eval jobs passing for every topology; collect-evals/collect-results/compare-results all succeeded.
  • Both config keys have evidence: the 5 refreshed topologies (new image) and the split-out 2P×1D DEP4 c256x512 on the legacy image each have their own non-skipped benchmark and eval jobs, all green.
  • Chat template (spec decode): both the refreshed and legacy EAGLE3 recipes set use_chat_template: true with speculative-config {method: eagle3, model: Inferact/MiniMax-M3-EAGLE3-GQA}, aligning the AL distribution to real-world chat usage.
  • Legacy-image split: the DEP4 decode topology is intentionally kept on the prior image under a separate matrix key; both the refreshed and legacy tags are upstream vllm/vllm-openai nightlies.
  • No engine patching: changes are declarative Dynamo/srt-slurm recipe YAML; the pinned images run as shipped — no patch files, sed/heredoc rewrites, site-packages edits, or forked wheels.
  • Agentic item (unchecked): not applicable — fixed-seq-len (8k1k) multi-node throughput/latency configs, not agentic workloads; agentic jobs correctly skipped.
  • Single-node recipe item (unchecked): not applicable — this PR touches only multi-node DISAGG srt-slurm recipes; the single-node vLLM/SGLang recipe-documentation requirement is out of scope for a disaggregated multi-node submission.

Signed: Ankur-singh

@Klaud-Cold

Copy link
Copy Markdown
Collaborator

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — Ankur-singh is a named owner of configs/nvidia-master.yaml; the recipe/changelog paths fall under the * catch-all, covered by any recognized CODEOWNER.
✅ Check 1 (green sweep on in-PR commit): PASS — head b1c771c5 (still the PR tip) carries run 30042427571 with all six multi-node 8k1k / and all six multi-node eval / check-runs success (executed, non-skipped; this PR has no single-node lanes).
✅ Check 2 (evals pass): PASS — gsm8k em_strict 0.953–0.958 across all six topologies vs the 0.90 bar, on the exact PR images (nightly-4080263… for the 5 refreshed, legacy nightly-8e98163… for DEP4). Informational: attempt 1's DEP4 eval scored 0.842 (below bar) and was rerun; the final green attempt scored 0.9545.
➖ Check 3 (upstream recipe link): N/A — exclusively multi-node DISAGG (srt-slurm recipes; multinode: true, disagg: true, dynamo-vllm); the recipe-link requirement covers single-node recipes only.
✅ Check 4 (reuse command): PASS — /reuse-sweep-run posted by xinli-sw (COLLABORATOR).
✅ Check 5 (latest checklist): PASS — every current-template item present; the two unchecked items (agentic AL, single-node recipe) are explained as N/A in the additional detail section.
✅ Check 6 (upstream image / engine-first): PASS — both config keys pin upstream vllm/vllm-openai nightlies; vLLM-first for minimaxm3+B300 already satisfied by minimaxm3-fp4-b300-vllm / minimaxm3-fp8-b300-vllm.
✅ Check 7 (no architecture hacks): PASS — diff only swaps image tags, drops the deprecated VLLM_FLASHINFER_ALLREDUCE_BACKEND env, and splits a matrix key; no FLOPs-reducing overrides.
✅ Check 8 (spec-decode chat template): PASS — all six EAGLE3 recipes set use_chat_template: true.
✅ Check 9 (no engine patches): PASS — declarative recipe/config YAML only; pinned images run as shipped.
➖ Check 10 (agentic golden AL): N/A — no agentic spec-decode changes (fixed-seq-len 8k1k), and no synthetic-acceptance knobs on these non-agentic configs.

@faradawn

faradawn commented Jul 24, 2026

Copy link
Copy Markdown

recipe. vllm-project/recipes#682

@Oseltamivir
Oseltamivir merged commit 37ead2a into main Jul 26, 2026
26 checks passed
@Oseltamivir
Oseltamivir deleted the agent/refresh-minimax-m3-b300-eagle branch July 26, 2026 14:21
Oseltamivir added a commit that referenced this pull request Jul 26, 2026
check-changelog rejected the previous push: main had appended the MiniMax-M3
NVFP4 B300 entry (#2314) after this branch's last merge, so relative to current
main the branch looked like it deleted those 19 lines.

Resolved per KLAUD_DEBUG.md 1.1 -- perf-changelog.yaml is rebuilt as current
main's bytes verbatim plus this branch's own five entries re-appended at the
tail, verified byte-exact. No 3-way merge of that file: it silently rewrites
whitespace and re-trips the deletion check.

中文:上一次推送被 check-changelog 拒绝:本分支上次合并之后,main 追加了
MiniMax-M3 NVFP4 B300 条目(#2314),因此相对于当前 main,本分支看起来删除了那 19 行。

已按 KLAUD_DEBUG.md 1.1 处理:perf-changelog.yaml 重建为当前 main 的原始字节,再将本
分支自身的五个条目重新追加到文件末尾,并校验为字节级纯追加。该文件不做三方合并——
三方合并会静默改动空白字符并再次触发删除检查。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

non-canary-full-sweep-enabled Run the full sweep without the canary gate (full search space, no trim)

Projects

Development

Successfully merging this pull request may close these issues.

6 participants