Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion benchmarks/single_node/agentic/minimaxm3_fp4_b300_mtp.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ install_agentic_deps

OFFLOAD_ARGS=()
if require_agentic_kv_offload_backend vllm-simple; then
CPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1024 * 1024 * 1024))
CPU_OFFLOAD_BYTES=$((TOTAL_CPU_DRAM_GB * 1000 * 1000 * 1000))
export VLLM_USE_SIMPLE_KV_OFFLOAD=1
OFFLOAD_CONFIG=$(printf \
'{"kv_connector":"SimpleCPUOffloadConnector","kv_role":"kv_both","kv_connector_extra_config":{"cpu_bytes_to_use":%d,"lazy_offload":true}}' \
Expand Down
15 changes: 9 additions & 6 deletions configs/nvidia-master.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7395,7 +7395,7 @@ qwen3.5-fp4-gb200-dynamo-sglang-agentic-mtp:
decode: { num-worker: 0, tp: 2, ep: 2, dp-attn: false }

minimaxm3-fp4-b300-vllm-agentic-mtp:
image: vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7
image: vllm/vllm-openai:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9
model: nvidia/MiniMax-M3-NVFP4
model-prefix: minimaxm3
runner: cluster:b300-nv
Expand All @@ -7404,13 +7404,16 @@ minimaxm3-fp4-b300-vllm-agentic-mtp:
multinode: false
scenarios:
agentic-coding:
# 0.683 maps the B300 TP4 SimpleCPU point to a 1,024 GB
# engine-level CPU KV budget. GPU-resident points receive a zero budget.
# 0.683 maps the B300 TP4 SimpleCPU point to a 1,024 GB engine-level CPU
# KV budget. TP2 uses 1.0 so 2 of 8 GPUs receive the 750 GB policy maximum.
- dram-utilization: 0.683
search-space:
- { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 2, 5, 10, 15, 20] }
- { tp: 2, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 2, 5] }
- { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [30, 40, 50, 60, 65, 70, 75] }
- { tp: 8, spec-decoding: mtp, kv-offloading: none, conc-list: [1] }
- { tp: 4, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 5, 10, 15, 20] }
- { tp: 4, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [30] }
- dram-utilization: 1.0
search-space:
- { tp: 2, spec-decoding: mtp, kv-offloading: dram, kv-offload-backend: { name: vllm-simple }, conc-list: [24] }
Comment thread
Ankur-singh marked this conversation as resolved.
Resolved

# GB200 DeepSeek-V4 disaggregated AgentX frontier. The 3P/2D TEP8/TP8 curve
# covers the middle/high-interactivity range omitted by the one-decode DEP
Expand Down
8 changes: 8 additions & 0 deletions perf-changelog.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5971,3 +5971,11 @@
- "Compare the B200 TP8 and DEP8 concurrency range and extend throughput with 1P1D and 2P1D DEP8 disaggregation."
- "Use KV-aware Dynamo routing with 4-hour correlation-ID affinity, authoritative vLLM KV events, KV-cache token metrics, and every logical vLLM server-metrics endpoint."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2567

- config-keys:
- minimaxm3-fp4-b300-vllm-agentic-mtp
scenario-type:
- agentic-coding
description:
- "Refresh the MiniMax-M3 B300 AgentX submission as eight EAGLE3-GQA points: no-offload TP8 C1 and TP4 C1/C5/C10/C15/C20, TP4 C30 with 1,024 GB SimpleCPU KV offload, and TP2 C24 with the proportional-policy 750 GB SimpleCPU KV-offload cap."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2610
Loading