[BugFix]: Fix DeepSeek V4 HiCache layer count logic - #25477
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the hierarchical cache layer mapping to use local offsets and relocates unified radix cache tests for Mamba and DeepSeek-V4 models from the nightly suite to the base CI. Feedback highlights a critical bug in hybrid_pool_assembler.py where local indices are used to access global state pools, which would cause incorrect behavior in Pipeline Parallelism. Additionally, the new test classes contain duplicate method definitions and empty overrides that silently disable inherited test cases, requiring cleanup to ensure the tests actually run.
|
/rerun-stage base-c |
|
Stage granularity is too coarse — a stage usually doesn't map to one feature, so rerunning a stage re-pays the cost of unrelated tests. If you don't know which exact test files to rerun, you shouldn't be using Use one of these instead:
AMD CI: stage-level dispatch is still available via Actions UI → PR Test (AMD) / PR Test ROCm 7.2 (AMD) → Run workflow → pick a stage from the dropdown. |
|
/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py |
|
🚀 |
|
/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py |
|
🚀 |
ShangmingCai
left a comment
There was a problem hiding this comment.
LGTM, let's wait for the CI.
|
/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl.py test/registered/radix_cache/test_unified_radix_cache_kl_hicache.py |
|
/rerun-test test/registered/radix_cache/test_unified_radix_cache_kl_hicache_nightly.py |
|
🚀 🚀 |
|
⛔ |
|
let's waiting for this ci: https://github.com/sgl-project/sglang/actions/runs/25964653117 |
Motivation
A previous PP support commit changed DeepSeekV4TokenToKVPool.layer_mapping to be stage-aware, leaving entries outside the active layer range as None. #24704
This PR updates the HiCache attach logic to use the actual active layer range, kvcache.end_layer -
kvcache.start_layer, and build sidecar mappings only from layer_mapping[start_layer:end_layer]. This aligns
HiCache transfer layers with real model layers.
CC @whybeyoung @ShangmingCai
Modifications
2.Move back some nightly ci to per-commit ci
Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): Run #25964626565⚠️ Not enabled — add
Latest PR Test (Extra):
run-ci-extralabel to opt in.