Skip to content

dsv4 sglang agentic: enable out-of-window SWA slot release and bump image to nightly-20260707 #2112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cquil11 merged 10 commits into from
Jul 9, 2026
Merged

dsv4 sglang agentic: enable out-of-window SWA slot release and bump image to nightly-20260707 #2112

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
be39b9f
dsv4 sglang agentic: enable out-of-window SWA slot release + bump ima…
Oasis-Git Jul 7, 2026
66664b2
Merge branch 'main' into reopen/pr-2111-sglang-swa
cquil11 Jul 7, 2026
1a04cd7
fix(runner): b200-dgxc ran sglang dsv4 as node-local NVFP4 instead of…
Oasis-Git Jul 8, 2026
7f60a2f
Merge branch 'main' into reopen/pr-2111-sglang-swa
Oasis-Git Jul 8, 2026
ff6b37b
Merge branch 'main' into reopen/pr-2111-sglang-swa
cquil11 Jul 8, 2026
3c26c6c
Add perf-changelog entry for dsv4 agentic sglang image bump and SWA e…
github-actions[bot] Jul 8, 2026
e6265cd
Merge branch 'main' into reopen/pr-2111-sglang-swa
cquil11 Jul 9, 2026
c699aab
Merge branch 'main' into reopen/pr-2111-sglang-swa
cquil11 Jul 9, 2026
e537315
Update perf-changelog.yaml
cquil11 Jul 9, 2026
7950816
Merge branch 'main' into reopen/pr-2111-sglang-swa
cquil11 Jul 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion benchmarks/single_node/agentic/dsv4_fp4_b200_sglang.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ install_agentic_deps
SERVER_LOG="$RESULT_DIR/server.log"
mkdir -p "$RESULT_DIR"

export SGLANG_ENABLE_UNIFIED_RADIX_TREE=1
export SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1

CACHE_ARGS=()
if require_agentic_kv_offload_backend hicache; then
# DeepSeek V4 HiCache currently rejects --hicache-size and supports
Expand All @@ -72,7 +75,6 @@ if require_agentic_kv_offload_backend hicache; then
HICACHE_WRITE_POLICY="${HICACHE_WRITE_POLICY:-write_through}"
HICACHE_IO_BACKEND="${HICACHE_IO_BACKEND:-direct}"
HICACHE_MEM_LAYOUT="${HICACHE_MEM_LAYOUT:-page_first_direct}"
export SGLANG_ENABLE_UNIFIED_RADIX_TREE=1
CACHE_ARGS=(
--enable-hierarchical-cache
--hicache-ratio "$HICACHE_RATIO"
Expand Down
4 changes: 3 additions & 1 deletion benchmarks/single_node/agentic/dsv4_fp4_b300_sglang.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,9 @@ install_agentic_deps
SERVER_LOG="$RESULT_DIR/server.log"
mkdir -p "$RESULT_DIR"

export SGLANG_ENABLE_UNIFIED_RADIX_TREE=1
export SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1

CACHE_ARGS=()
if require_agentic_kv_offload_backend hicache; then
# DeepSeek V4 HiCache currently rejects --hicache-size and supports
Expand All @@ -77,7 +80,6 @@ if require_agentic_kv_offload_backend hicache; then
HICACHE_WRITE_POLICY="${HICACHE_WRITE_POLICY:-write_back}"
HICACHE_IO_BACKEND="${HICACHE_IO_BACKEND:-direct}"
HICACHE_MEM_LAYOUT="${HICACHE_MEM_LAYOUT:-page_first_direct}"
export SGLANG_ENABLE_UNIFIED_RADIX_TREE=1
CACHE_ARGS=(
--enable-hierarchical-cache
--hicache-ratio "$HICACHE_RATIO"
Expand Down
4 changes: 2 additions & 2 deletions configs/nvidia-master.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12680,7 +12680,7 @@
- { tp: 8, ep: 8, kv-offloading: dram, kv-offload-backend: mooncake, conc-list: [5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 18, 20] }

dsv4-fp4-b200-sglang-agentic-hicache:
image: lmsysorg/sglang:v0.5.13-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260707-b4155233

Check failure on line 12683 in configs/nvidia-master.yaml

View check run for this annotation

Claude / Claude Code Review

Missing perf-changelog.yaml entry for image bump 

This PR bumps the SGLang image tag for `dsv4-fp4-b200-sglang-agentic-hicache` (line 12683) and `dsv4-fp4-b300-sglang-agentic-hicache` (line 12705) but does not append a matching entry to `perf-changelog.yaml`. Per AGENTS.md's "Updating Docker images" section, a changelog entry is required — it's what triggers `.github/workflows/run-sweep.yml` (which filters on `paths: - perf-changelog.yaml`). Without it, neither the PR fail-fast sweep nor the post-merge push-to-main sweep will actually run for t

@claude claude Bot Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This PR bumps the SGLang image tag for dsv4-fp4-b200-sglang-agentic-hicache (line 12683) and dsv4-fp4-b300-sglang-agentic-hicache (line 12705) but does not append a matching entry to perf-changelog.yaml. Per AGENTS.md's "Updating Docker images" section, a changelog entry is required — it's what triggers .github/workflows/run-sweep.yml (which filters on paths: - perf-changelog.yaml). Without it, neither the PR fail-fast sweep nor the post-merge push-to-main sweep will actually run for these two configs, so the ~40K→106K tok/s / TTFT 17s→6s improvements claimed in the PR description will not be reflected in tracked main-branch results.

Extended reasoning...

What the bug is

The PR modifies exactly 3 files: two launcher scripts (benchmarks/single_node/agentic/dsv4_fp4_b{200,300}_sglang.sh) and configs/nvidia-master.yaml. In configs/nvidia-master.yaml it bumps the image for two configs:

  • dsv4-fp4-b200-sglang-agentic-hicache at line 12683: lmsysorg/sglang:v0.5.13-cu130 → lmsysorg/sglang:nightly-dev-cu13-20260707-b4155233
  • dsv4-fp4-b300-sglang-agentic-hicache at line 12705: same bump

It also adds two new engine env vars (SGLANG_ENABLE_UNIFIED_RADIX_TREE=1 and SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1) to both launcher scripts. However, perf-changelog.yaml is not touched at all.

Why this matters — the trigger contract

AGENTS.md (Updating Docker images) is explicit: on an image bump you must "append a perf-changelog.yaml entry (required - triggers benchmarks)". This is not just documentation hygiene — it is the actual trigger:

# .github/workflows/run-sweep.yml
on:
  push:
    branches: [main]
    paths:
      - perf-changelog.yaml
  pull_request:
    paths:
      - perf-changelog.yaml

Because the workflow's paths: filter only fires when perf-changelog.yaml is modified, this PR's diff will not cause run-sweep.yml to invoke — despite the full-sweep-fail-fast label being set. The label controls behavior inside the workflow once it runs; it does not cause the workflow to run in the first place.

Concrete step-by-step proof

  1. Contributor pushes commits touching configs/nvidia-master.yaml and the two .sh files.
  2. GitHub Actions receives the pull_request event.
  3. Actions consults run-sweep.yml's on.pull_request.paths filter: perf-changelog.yaml — not in the changed file list.
  4. Actions does not schedule the sweep workflow. No matrix job runs; no fail-fast gate exists.
  5. PR merges (nothing gated it). push to main fires; on.push.paths filter again requires perf-changelog.yaml — again not present.
  6. No post-merge sweep runs against these two configs on the new image. The tracked main-branch performance dashboards keep the old image's numbers.

Precedent — every recent bump followed this convention

Recent image-bump PRs (#2056, #2058, #2060, #2067, #2073, #2074, #2077) all touched both configs/*-master.yaml and appended a perf-changelog.yaml block. That's the workflow this repo relies on.

Fix

Append an entry to perf-changelog.yaml in the same style as the tail of that file — model+precision+SKU, PR number, one-line reason (e.g. "Bump sglang nightly to pick up unified radix-tree out-of-window SWA slot release"), and the two affected config keys (dsv4-fp4-b200-sglang-agentic-hicache and dsv4-fp4-b300-sglang-agentic-hicache). That single addition arms both the PR fail-fast sweep and the post-merge push-to-main sweep.

model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: cluster:b200-dgxc
Expand All @@ -12702,7 +12702,7 @@
# covers the middle/high-interactivity range omitted by the one-decode DEP
# throughput curves below. Each engine start carries at most four concurrencies.
dsv4-fp4-b300-sglang-agentic-hicache:
image: lmsysorg/sglang:v0.5.13-cu130
image: lmsysorg/sglang:nightly-dev-cu13-20260707-b4155233
model: deepseek-ai/DeepSeek-V4-Pro
model-prefix: dsv4
runner: cluster:b300-nv
Expand Down
Loading