Skip to content

Update DSV4 B300 SGLang AgentX image and HiCache concurrency grid / 更新 DSV4 B300 SGLang AgentX 镜像和 HiCache 并发网格 #2759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 5 commits into from
Sep 1, 2026
+11 −2
Merged

Update DSV4 B300 SGLang AgentX image and HiCache concurrency grid / 更新 DSV4 B300 SGLang AgentX 镜像和 HiCache 并发网格 #2759

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
231f654
chore(dsv4-b300): move AgentX HiCache MTP lane to a published nightly…
yhyang201 Aug 27, 2026
4960d11
chore(changelog): point the entry at PR #2759
yhyang201 Aug 27, 2026
705eb60
fix(dsv4-b300): drop DEP8 concurrency 64 and 128 from the hicache row
yhyang201 Aug 28, 2026
fb85931
Merge origin/main into dsv4-b300-sglang-nightly-20260827
yhyang201 Aug 28, 2026
1416480
docs(perf): consolidate PR 2759 changelog [skip-sweep]
cquil11 Sep 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1155,7 +1155,7 @@ dsv4-fp4-b300-sglang:
- { tp: 8, ep: 8, dp-attn: true, conc-start: 4096, conc-end: 4096 }

dsv4-fp4-b300-sglang-agentic-hicache-mtp:
image: lmsysorg/sglang-staging:dev-cu13-pr-35880
image: lmsysorg/sglang:nightly-dev-cu13-20260827-20621aa1
Comment thread
cquil11 marked this conversation as resolved.
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: cluster:b300-nv
Expand All @@ -1167,7 +1167,7 @@ dsv4-fp4-b300-sglang-agentic-hicache-mtp:
- dram-utilization: 0.95
search-space:
- { tp: 8, kv-offloading: none, spec-decoding: mtp, conc-list: [1, 4, 8, 16, 32] }
- { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, spec-decoding: mtp, conc-list: [32, 64, 128, 256, 384, 512, 576], router: { name: sglang-router, version: "0.3.2" } }
- { tp: 8, ep: 8, dp-attn: true, kv-offloading: dram, kv-offload-backend: { name: hicache }, spec-decoding: mtp, conc-list: [32, 256, 384, 512, 576], router: { name: sglang-router, version: "0.3.2" } }

# DeepSeek-V4-Pro on B300 with EAGLE/MTP speculative decoding. Recipe is
# selected inside benchmarks/single_node/dsv4_fp4_b300_sglang_mtp.sh by
Expand Down
9 changes: 9 additions & 0 deletions perf-changelog.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6741,3 +6741,12 @@
- "Configure model-specific tool and reasoning parsers for the Kimi-K3 and MiniMax-M3 serving paths so opt-in verifier requests produce structured OpenAI tool calls."
- "Limit shared AgentX readiness and synthetic-acceptance rewrites to eval-only runs so default throughput recipe rendering remains unchanged."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2634

- config-keys:
- dsv4-fp4-b300-sglang-agentic-hicache-mtp
scenario-type:
- agentic-coding
description:
- "Update SGLang from lmsysorg/sglang-staging:dev-cu13-pr-35880 to lmsysorg/sglang:nightly-dev-cu13-20260827-20621aa1."
- "Remove DEP8 HiCache concurrency 64 and 128: 128 ran out of GPU memory at mem-fraction-static=0.93; 64 was unmeasured and used the same memory budget."
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2759