Skip to content

[AgentX] MiniMax-M3.1-Flash GB300 SGLang: add TP2 configs / [AgentX] MiniMax-M3.1-Flash GB300 SGLang：新增 TP2 配置 #4511

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
edwingao28 merged 1 commit into from
Oct 26, 2026
Merged

[AgentX] MiniMax-M3.1-Flash GB300 SGLang: add TP2 configs / [AgentX] MiniMax-M3.1-Flash GB300 SGLang：新增 TP2 配置 #4511

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions ...enchmarks/single_node/srt-slurm-recipes/minimaxm31flash/sglang/gb300-fp4-mtp/agentic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ override_tp4_hicache_c64:
chunked-prefill-size: 32768
max-running-requests: 128
cuda-graph-max-bs-decode: 128
mem-fraction-static: 0.8
mem-fraction-static: 0.87
enable-hierarchical-cache: true
hicache-size: 180
hicache-write-policy: write_back
Expand All @@ -87,6 +87,28 @@ override_tp4_hicache_c64:
CONC: '64'
KV_OFFLOADING: dram

# TP2 with L2 HiCache: two-GPU engines at 180 GB per rank, inside their share of the
# node's step memory cap. The scheduler batch is 2x CONC and the decode graph batch matches it.

zip_override_tp2_hicache:
roles:
agg:
gpus: 2
args:
tensor-parallel-size: 2
moe-runner-backend: flashinfer_trtllm_routed
chunked-prefill-size: 32768
max-running-requests: [64, 80, 88]
cuda-graph-max-bs-decode: [64, 80, 88]
mem-fraction-static: 0.87
enable-hierarchical-cache: true
hicache-size: 180
hicache-write-policy: write_back
benchmark:
env:
CONC: ['32', '40', '44']
KV_OFFLOADING: dram

# High-Throughput cell: TP4 with EP4 (MegaMoE). The scheduler batch and decode graph
# batch are the cookbook's 128. Above conc 32 L2 HiCache spills KV to host DRAM; the
# per-rank pool keeps all four ranks inside the node's 920,000 MiB step memory cap.
Expand Down Expand Up @@ -117,7 +139,7 @@ override_tp4_ep4_hicache_c64:
chunked-prefill-size: 32768
max-running-requests: 128
cuda-graph-max-bs-decode: 128
mem-fraction-static: 0.8
mem-fraction-static: 0.87
enable-hierarchical-cache: true
hicache-size: 175
hicache-write-policy: write_back
Expand All @@ -137,7 +159,7 @@ override_tp4_ep4_hicache_c128:
chunked-prefill-size: 32768
max-running-requests: 128
cuda-graph-max-bs-decode: 128
mem-fraction-static: 0.8
mem-fraction-static: 0.87
enable-hierarchical-cache: true
hicache-size: 175
hicache-write-policy: write_back
Expand Down
2 changes: 2 additions & 0 deletions inferencex-e2e/configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8954,6 +8954,8 @@ minimaxm31flash-fp4-gb300-sglang-agentic-dspark:
search-space:
# Balanced TP4 with L2 HiCache: 180 GB per rank at conc 64.
- { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [64], srt-recipe: agentic.yaml }
# TP2 with L2 HiCache: 180 GB per rank at conc 32-44.

@claude claude Bot Oct 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 nit (optional): The new comments label the TP2 row and override, or restate values the YAML already shows, which AGENTS.md's mandatory comment rules forbid. sweep:# TP2 with L2 HiCache Example: nvidia-master.yaml:8957 only repeats tp 2 and conc-list [32, 40, 44]. Fix: drop the label and restated values, and keep only the reason, such as why 180 GB per rank fits the two-GPU DRAM allocation.

Why this was flagged

AGENTS.md:56 bans header comments that describe a config, override or section. AGENTS.md:58 allows only technical reasons for a line and says never restate the code. The new comment at nvidia-master.yaml:8957 says what the next line already encodes: tp 2, 180 GB, conc 32-44. The new block at agentic.yaml:90-91 opens with "TP2 with L2 HiCache: two-GPU engines", which labels zip_override_tp2_hicache. It also restates gpus: 2 and the 2x CONC batch. Neighbouring overrides use the same style, but that is precedent, not an exemption. No runtime effect.

Verification: AGENTS.md:54-58 forbids "header comments describing a ... config, recipe, override, or section" and says "never restate the code". nvidia-master.yaml:8957 adds # TP2 with L2 HiCache: 180 GB per rank at conc 32-44., which labels the row and restates tp: 2 and conc-list: [32, 40, 44]. agentic.yaml:90-91 labels zip_override_tp2_hicache and restates gpus: 2. Nothing breaks at runtime.

- { tp: 2, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [32, 40, 44], srt-recipe: agentic.yaml }
# High-Throughput with L2 HiCache: 175 GB per rank at conc 64 and 128.
- { tp: 4, ep: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: hicache }, conc-list: [64, 128], srt-recipe: agentic.yaml }

Expand Down
7 changes: 7 additions & 0 deletions inferencex-e2e/perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11197,3 +11197,10 @@
- "Balanced TP4 at conc 64 with L2 HiCache (write_back, hicache-size 180 GB per rank), and the scheduler and decode graph batch at 128."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/4499
scenario-type: [agentic-coding]

- config-keys:
- minimaxm31flash-fp4-gb300-sglang-agentic-dspark
description:
- "Add TP2 points with DRAM KV offload, and adjust the memory settings on the DRAM KV offload points."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX
scenario-type: [agentic-coding]
Loading