Repository navigation
[mem_cache] Replace is_chunk_cache / is_tree_cache with supports_prefix_sharing - #42362
Merged
Merged
Conversation
hnyls2002
requested review from
Ying1123,
alphabetc1,
hanming-lu,
huangtingwei9988,
hzh0425,
iforgetmyname,
ispobock,
merrymercy,
ping1jing2,
whybeyoung,
xiezhq-hermann and
yizhang2077
as code owners
October 3, 2026 07:46
github-actions
Bot
added
hicache
Collaborator
Author
|
/rerun-test test_gemma4_unified_swa_virtual_ids.py test_swa_radix_cache_kl.py test_streaming_session_swa.py test_mamba2_extra_buffer_kl.py |
Contributor
|
Results for 🚀 |
ispobock
approved these changes
Oct 3, 2026
Collaborator
Author
|
/tag-and-rerun-ci |
4 of 5 tasks
nvpohanh
added a commit
to peilii/sglang
that referenced
this pull request
Oct 5, 2026
Resolve conflicts with main: - schedule_policy.py: fold main's new max_prefix_len cap into _req_radix_key, and gate the refresh on supports_prefix_sharing() since main removed is_chunk_cache() (sgl-project#42362). - unified_radix_cache.py: keep refresh_device_prefix next to main's supports_prefix_sharing(). - streaming_session.py: take main's version. StreamingSession is no longer a BasePrefixCache wrapper (sgl-project#42295), so the refresh_device_prefix forwarder and its wrapper test are dropped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wangwenmingaa
pushed a commit
to wangwenmingaa/sglang
that referenced
this pull request
Oct 8, 2026
Keep the eviction-headroom reserve from this PR while adopting upstream sgl-project#39479, which switched the unified hybrid-SWA call site from evict_for_alloc() to evict() so cumulative reclaim quotas are fully honored. Update the unit-test assertion accordingly and refresh the mocked tree-cache guard renamed by sgl-project#42362 (is_chunk_cache -> supports_prefix_sharing).
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
BasePrefixCache.is_chunk_cache()/is_tree_cache()with one capability,supports_prefix_sharing(): whether a request's prefix stays in the cache for other requests to share, including after the request finishes.UnifiedRadixCachereturnsnot self.disable, so a disabled unified tree no longer reads as "not a tree cache".free_swa_out_of_window_slots(is_chunk_cache=False)becomessupports_prefix_sharing=True.RadixCachewithdisable=Trueand the remainingtree_cache.disablereaders are left as is.CI States
Latest PR Test (Base): 🚫 Run #37107458338
Latest PR Test (Extra): ❌ Run #37107458300
Latest PR Test (AMD ROCm 10): ❌ Run #37107458384