Repository navigation
[AgentX] MiniMax-M3.1-Flash B200/GB200/B300 SGLang: add conc 1 low-latency configs / [AgentX] MiniMax-M3.1-Flash B200/GB200/B300 SGLang：新增 conc 1 低延迟配置 #4509
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
4a2f670
Add low-latency conc 1 to MiniMax-M3.1-Flash B200, GB200 and B300 SGL…
ispobock 16e03c7
Drop the derivation note from the low-latency recipe comments
ispobock faf5de6
Merge remote-tracking branch 'origin/main' into minimaxm31flash-sglan…
ispobock c9d58f9
Merge remote-tracking branch 'origin/main' into pr-4509
zijiexia 20a3067
Merge remote-tracking branch 'origin/main' into minimaxm31flash-sglan…
ispobock 4231e37
Merge remote-tracking branch 'origin/main' into minimaxm31flash-sglan…
ispobock 0d59a67
Merge remote-tracking branch 'origin/main' into minimaxm31flash-sglan…
ispobock File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
64 changes: 64 additions & 0 deletions
64
...ingle_node/srt-slurm-recipes/minimaxm31flash/sglang/b200-fp4-mtp/agentic-low-latency.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| base: | ||
| schema: 2 | ||
| name: minimaxm31flash-fp4-b200-sglang-agentic-low-latency | ||
| resources: | ||
| gpu_type: b200 | ||
| gpus_per_node: 8 | ||
| frontend: | ||
| type: sglang | ||
| enable_multiple_frontends: false | ||
| observability: | ||
| enabled: false | ||
| tachometer: | ||
| enabled: false | ||
| engine: sglang | ||
| # Cold loads of the 250 GB checkpoint plus JIT MSA kernel builds and graph capture outlast the default. | ||
| health_check: | ||
| max_attempts: 1440 | ||
| interval_seconds: 5 | ||
| roles: | ||
| agg: | ||
| nodes: 1 | ||
| workers: 1 | ||
| gpus: 4 | ||
| # Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1 | ||
| # DSpark steps; larger batches take the standard path. | ||
| args: | ||
| served-model-name: MiniMaxAI/MiniMax-M3.1-Flash | ||
| trust-remote-code: true | ||
| reasoning-parser: minimax-m3 | ||
| tool-call-parser: minimax-m3 | ||
| tensor-parallel-size: 4 | ||
| moe-runner-backend: flashinfer_trtllm_routed | ||
| page-size: 128 | ||
| speculative-algorithm: DSPARK | ||
| speculative-use-block-verification: true | ||
| speculative-num-steps: 1 | ||
| speculative-eagle-topk: 1 | ||
| speculative-num-draft-tokens: 8 | ||
| speculative-dspark-block-size: 7 | ||
| monokernel-backend: msa | ||
| disable-attn-tp-gather: true | ||
| max-running-requests: 8 | ||
| mem-fraction-static: 0.8 | ||
| watchdog-timeout: 3600 | ||
| enable-metrics: true | ||
| env: | ||
| PYTHONNOUSERSITE: '1' | ||
| PYTHONUNBUFFERED: '1' | ||
| SGLANG_TIMEOUT_KEEP_ALIVE: '1800' | ||
| FMHA_SM100_ALLOW_JIT: '1' | ||
| ALLOW_JIT: '1' | ||
| # Canonical MegaAttention placement; it needs the GPUs to itself. | ||
| SGLANG_FORCE_COARSE_WAR_BARRIER: '1' | ||
| benchmark: | ||
| env: | ||
| # Warmup can leave bytes unacknowledged past AIPerf's 30 s default. | ||
| AIPERF_HTTP_TCP_USER_TIMEOUT: '900000' | ||
| AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'sglang:' | ||
|
|
||
| override_tp4_c1: | ||
| benchmark: | ||
| env: | ||
| CONC: '1' | ||
| KV_OFFLOADING: none | ||
64 changes: 64 additions & 0 deletions
64
...ingle_node/srt-slurm-recipes/minimaxm31flash/sglang/b300-fp4-mtp/agentic-low-latency.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| base: | ||
| schema: 2 | ||
| name: minimaxm31flash-fp4-b300-sglang-agentic-low-latency | ||
| resources: | ||
| gpu_type: b300 | ||
| gpus_per_node: 8 | ||
| frontend: | ||
| type: sglang | ||
| enable_multiple_frontends: false | ||
| observability: | ||
| enabled: false | ||
| tachometer: | ||
| enabled: false | ||
| engine: sglang | ||
| # Cold loads of the 250 GB checkpoint plus JIT MSA kernel builds and graph capture outlast the default. | ||
| health_check: | ||
| max_attempts: 1440 | ||
| interval_seconds: 5 | ||
| roles: | ||
| agg: | ||
| nodes: 1 | ||
| workers: 1 | ||
| gpus: 4 | ||
| # Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1 | ||
| # DSpark steps; larger batches take the standard path. | ||
| args: | ||
| served-model-name: MiniMaxAI/MiniMax-M3.1-Flash | ||
| trust-remote-code: true | ||
| reasoning-parser: minimax-m3 | ||
| tool-call-parser: minimax-m3 | ||
| tensor-parallel-size: 4 | ||
| moe-runner-backend: flashinfer_trtllm_routed | ||
| page-size: 128 | ||
| speculative-algorithm: DSPARK | ||
| speculative-use-block-verification: true | ||
| speculative-num-steps: 1 | ||
| speculative-eagle-topk: 1 | ||
| speculative-num-draft-tokens: 8 | ||
| speculative-dspark-block-size: 7 | ||
| monokernel-backend: msa | ||
| disable-attn-tp-gather: true | ||
| max-running-requests: 8 | ||
| mem-fraction-static: 0.8 | ||
| watchdog-timeout: 3600 | ||
| enable-metrics: true | ||
| env: | ||
| PYTHONNOUSERSITE: '1' | ||
| PYTHONUNBUFFERED: '1' | ||
| SGLANG_TIMEOUT_KEEP_ALIVE: '1800' | ||
| FMHA_SM100_ALLOW_JIT: '1' | ||
| ALLOW_JIT: '1' | ||
| # Canonical MegaAttention placement; it needs the GPUs to itself. | ||
| SGLANG_FORCE_COARSE_WAR_BARRIER: '1' | ||
| benchmark: | ||
| env: | ||
| # Warmup can leave bytes unacknowledged past AIPerf's 30 s default. | ||
| AIPERF_HTTP_TCP_USER_TIMEOUT: '900000' | ||
| AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'sglang:' | ||
|
|
||
| override_tp4_c1: | ||
| benchmark: | ||
| env: | ||
| CONC: '1' | ||
| KV_OFFLOADING: none |
64 changes: 64 additions & 0 deletions
64
...ngle_node/srt-slurm-recipes/minimaxm31flash/sglang/gb200-fp4-mtp/agentic-low-latency.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| base: | ||
| schema: 2 | ||
| name: minimaxm31flash-fp4-gb200-sglang-agentic-low-latency | ||
| resources: | ||
| gpu_type: gb200 | ||
| gpus_per_node: 4 | ||
| frontend: | ||
| type: sglang | ||
| enable_multiple_frontends: false | ||
| observability: | ||
| enabled: false | ||
| tachometer: | ||
| enabled: false | ||
| engine: sglang | ||
| # Cold loads of the 250 GB checkpoint plus JIT MSA kernel builds and graph capture outlast the default. | ||
| health_check: | ||
| max_attempts: 1440 | ||
| interval_seconds: 5 | ||
| roles: | ||
| agg: | ||
| nodes: 1 | ||
| workers: 1 | ||
| gpus: 4 | ||
| # Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1 | ||
| # DSpark steps; larger batches take the standard path. | ||
| args: | ||
| served-model-name: MiniMaxAI/MiniMax-M3.1-Flash | ||
| trust-remote-code: true | ||
| reasoning-parser: minimax-m3 | ||
| tool-call-parser: minimax-m3 | ||
| tensor-parallel-size: 4 | ||
| moe-runner-backend: flashinfer_trtllm_routed | ||
| page-size: 128 | ||
| speculative-algorithm: DSPARK | ||
| speculative-use-block-verification: true | ||
| speculative-num-steps: 1 | ||
| speculative-eagle-topk: 1 | ||
| speculative-num-draft-tokens: 8 | ||
| speculative-dspark-block-size: 7 | ||
| monokernel-backend: msa | ||
| disable-attn-tp-gather: true | ||
| max-running-requests: 8 | ||
| mem-fraction-static: 0.8 | ||
| watchdog-timeout: 3600 | ||
| enable-metrics: true | ||
| env: | ||
| PYTHONNOUSERSITE: '1' | ||
| PYTHONUNBUFFERED: '1' | ||
| SGLANG_TIMEOUT_KEEP_ALIVE: '1800' | ||
| FMHA_SM100_ALLOW_JIT: '1' | ||
| ALLOW_JIT: '1' | ||
| # Canonical MegaAttention placement; it needs the GPUs to itself. | ||
| SGLANG_FORCE_COARSE_WAR_BARRIER: '1' | ||
| benchmark: | ||
| env: | ||
| # Warmup can leave bytes unacknowledged past AIPerf's 30 s default. | ||
| AIPERF_HTTP_TCP_USER_TIMEOUT: '900000' | ||
| AIPERF_REQUIRED_SERVER_METRIC_PREFIX: 'sglang:' | ||
|
|
||
| override_tp4_c1: | ||
| benchmark: | ||
| env: | ||
| CONC: '1' | ||
| KV_OFFLOADING: none |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 nit (optional): the new recipes add a comment saying where the config came from, which AGENTS.md's "No history" rule bans. sweep:
# Follows the GB300 Low-Latency cellThe comment aboveargs:says what the recipe was copied from. The rule says derivation belongs in git and perf-changelog. Fix: drop "Follows the GB300 Low-Latency cell:" from each of the three new recipes. Keep only the technical part, for example# MSA-V1 MonoKernel serves batch-size-1 DSpark steps; larger batches take the standard path.Why this was flagged
The three new recipe files b200-fp4-mtp/agentic-low-latency.yaml:24, b300-fp4-mtp/agentic-low-latency.yaml:24 and gb200-fp4-mtp/agentic-low-latency.yaml:24 each add
# Follows the GB300 Low-Latency cell: the MSA-V1 MonoKernel serves batch-size-1. AGENTS.md "Code comments (mandatory)" covers recipes and says: "Do not write comments that describe how the code evolved, where it was derived from, what it replaced". The opening clause describes where the recipe was derived from, not why a value was chosen. Behaviour does not change. The cost is a convention breach that leaves maintainers a stale pointer once the GB300 recipe changes.Verification: Each new recipe (b200-fp4-mtp, b300-fp4-mtp and gb200-fp4-mtp
agentic-low-latency.yaml) adds the same two lines at 24-25, just aboveargs:, starting "# Follows the GB300 Low-Latency cell". Line 65 of AGENTS.md says: "No history. Do not write comments that describe... where it was derived from". That clause is exactly what that rule bans. No runtime behavior is affected.