[PP] Fix start_layer_id with pp in get kv_buffer_shape - #29887
Merged
Merged
Conversation
staugust
requested review from
Fridge003,
Ying1123,
hnyls2002,
ispobock and
merrymercy
as code owners
July 2, 2026 03:15
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Collaborator
|
/tag-and-rerun-ci |
whybeyoung
reviewed
Jul 2, 2026
whybeyoung
approved these changes
Jul 2, 2026
staugust
requested review from
alphabetc1,
hanming-lu,
hzh0425,
xiezhq-hermann and
yizhang2077
as code owners
July 2, 2026 04:08
Collaborator
|
/tag-and-rerun-ci |
whybeyoung
changed the title
ShangmingCai
approved these changes
Jul 6, 2026
Collaborator
|
/rerun-failed-ci |
Collaborator
|
/rerun-test test/registered/models_e2e/test_mimo_v2.py |
Contributor
|
Results for 🚀 |
5 tasks
Collaborator
|
/rerun-failed-ci |
ShangmingCai
changed the title
5 tasks
hzwzwzw
added a commit
to hzwzwzw/sglang
that referenced
this pull request
Jul 13, 2026
…last-leaf fix Upstream sgl-project#29860 (a375e9f, merged). Adapted to our fork's inline _evict_swa in schedule_batch.py (upstream lives in mem_cache/common.py:free_swa_out_of_window_slots). Before: _evict_swa evicted up to pre_len - sliding_window_size - page_size under the assumption that "extra page keeps the frontier below the insert boundary". The env flag SGLANG_OPT_SWA_EVICT_DROP_PAGE_MARGIN toggled whether to apply the extra -page_size subtraction. After: gate on tree_cache.is_chunk_cache() (replicated tree-cache property, uniform across ranks) instead of the env flag: - chunk-cache: no radix tree -> no tombstone-leaf concern; evict up to the window boundary (pre_len - sliding_window_size). - radix: keep max(window, page). The trailing floor page-aligns the frontier, and subtracting at least one page keeps the frontier below the insert boundary (page_floor(seq_len)) so the last leaf is never all-tombstone. This is the case sgl-project#29860 was fixing. The env var SGLANG_OPT_SWA_EVICT_DROP_PAGE_MARGIN is now unused but left in environ.py for backward compatibility (harmless no-op). P2 batch, other PRs skipped this round: - sgl-project#27550 fix(hiradix): wait for extra pool IO - target code path (the completed+pool_transfers_done branch in can_terminate_prefetch) was simplified away by our PR sgl-project#27010 port; the pool_transfers_done invariant is now enforced through the ack-queue ordering instead. - sgl-project#28422 decode-hicache _storage_hit_query - the "pre-query" feature sgl-project#28422 patches does not exist in our fork. - sgl-project#29887 [PP] get kv_buffer_shape - target file (eager_runner.py) does not exist in our fork. - sgl-project#29817 write_back policy refinement - refactor, not a bug fix; our fork's evict() has diverged from the upstream shape and porting cleanly is out of scope for this batch. - sgl-project#28614 remove large host mem constraint - depends on sgl-project#29817's evict refactor for the bulk of its diff, and the standalone piece flips prefetch_capacity_limit from max(0, 0.8*(host-device)) to 0.5*host, a subtle memory-budget semantics change we won't ship silently. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Chronostasys
pushed a commit
to MindLab-Research/sglang
that referenced
this pull request
Aug 24, 2026
…9887) Co-authored-by: ybyang <10629930+whybeyoung@users.noreply.github.com> Co-authored-by: Shangming Cai <csmthu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fix bugs for Deepseek V3.1/V3.2/Glm5/GLM5.1 and pp_size > 1, for pp ranks > 0, start _layer_id is not 0.
Fix #29877
Modifications
get_key_buffer from start_layer.
Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ciCI States
Latest PR Test (Base): ❌ Run #28799607482
Latest PR Test (Extra): ❌ Run #28799607823