Skip to content

[BugFix]: Fix DeepSeek V4 HiCache layer count logic - #25477

Merged
ShangmingCai merged 4 commits into
sgl-project:mainfrom
hzh0425:hybrid_tree/fix-v4-crash
May 16, 2026
Merged

[BugFix]: Fix DeepSeek V4 HiCache layer count logic#25477
ShangmingCai merged 4 commits into
sgl-project:mainfrom
hzh0425:hybrid_tree/fix-v4-crash

Conversation

@hzh0425

@hzh0425 hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator

Motivation

A previous PP support commit changed DeepSeekV4TokenToKVPool.layer_mapping to be stage-aware, leaving entries outside the active layer range as None. #24704

This PR updates the HiCache attach logic to use the actual active layer range, kvcache.end_layer -
kvcache.start_layer, and build sidecar mappings only from layer_mapping[start_layer:end_layer]. This aligns
HiCache transfer layers with real model layers.

CC @whybeyoung @ShangmingCai

Modifications

  1. Fix layer count logic
    2.Move back some nightly ci to per-commit ci

Accuracy Tests

Speed Tests and Profiling

Checklist

Review and Merge Process

  1. Ping Merge Oncalls to start the process. See the PR Merge Process.
  2. Get approvals from CODEOWNERS and other reviewers.
  3. Trigger CI tests with comments or contact authorized users to do so.
    • Common commands include /tag-and-rerun-ci, /tag-run-ci-label, /rerun-failed-ci
  4. After green CI and required approvals, ask Merge Oncalls or people with Write permission to merge the PR.

CI States

Latest PR Test (Base): Run #25964626565
Latest PR Test (Extra): ⚠️ Not enabled — add run-ci-extra label to opt in.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the hierarchical cache layer mapping to use local offsets and relocates unified radix cache tests for Mamba and DeepSeek-V4 models from the nightly suite to the base CI. Feedback highlights a critical bug in hybrid_pool_assembler.py where local indices are used to access global state pools, which would cause incorrect behavior in Pipeline Parallelism. Additionally, the new test classes contain duplicate method definitions and empty overrides that silently disable inherited test cases, requiring cleanup to ensure the tests actually run.

Comment thread python/sglang/srt/mem_cache/hybrid_cache/hybrid_pool_assembler.py
Comment thread test/registered/radix_cache/test_unified_radix_cache_kl.py Outdated
Comment thread test/registered/radix_cache/test_unified_radix_cache_kl.py Outdated
@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-stage base-c

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ /rerun-stage has been deprecated.

Stage granularity is too coarse — a stage usually doesn't map to one feature, so rerunning a stage re-pays the cost of unrelated tests. If you don't know which exact test files to rerun, you shouldn't be using /rerun-stage or /rerun-test in the first place.

Use one of these instead:

  • Selective tests (you know exactly which files to rerun):
    /rerun-test test_foo.py test_bar.py
    
  • Rerun only failed jobs:
    /rerun-failed-ci
    
  • Full CI rerun (with extra coverage): add the run-ci or run-ci-extra label and push a new commit (or use /tag-and-rerun-ci).

AMD CI: stage-level dispatch is still available via Actions UI → PR Test (AMD) / PR Test ROCm 7.2 (AMD)Run workflow → pick a stage from the dropdown.

@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py

@github-actions

github-actions Bot commented May 16, 2026

Copy link
Copy Markdown
Contributor

🚀 4-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/radix_cache/test_unified_radix_cache_kl.py

@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py

@github-actions

github-actions Bot commented May 16, 2026

Copy link
Copy Markdown
Contributor

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/test_unified_radix_cache_kl.py

@ShangmingCai ShangmingCai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, let's wait for the CI.

@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py test/registered/radix_cache/test_unified_radix_cache_kl_hicache.py

@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl_hicache_nightly.py

@github-actions

github-actions Bot commented May 16, 2026

Copy link
Copy Markdown
Contributor

🚀 4-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/test_unified_radix_cache_kl.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/test_unified_radix_cache_kl_hicache.py

@github-actions

github-actions Bot commented May 16, 2026

Copy link
Copy Markdown
Contributor

test/registered/radix_cache/test_unified_radix_cache_kl_hicache_nightly.py: No register_cuda_ci(runner_config=...) or register_cpu_ci() found in test/registered/radix_cache/test_unified_radix_cache_kl_hicache_nightly.py. /rerun-test only supports tests registered via the new-style yml-driven API; nightly/weekly tests aren't dispatchable through this command.

@hzh0425

hzh0425 commented May 16, 2026

Copy link
Copy Markdown
Collaborator Author

@ShangmingCai
ShangmingCai merged commit 0be5390 into sgl-project:main May 16, 2026
97 of 120 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

hicache Hierarchical Caching for SGLang run-ci

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants