[mem_cache] Make free_swa sync-free on page_size == 1 - #36723
Merged
Merged
Conversation
hnyls2002
requested review from
ByronHsu,
Duyi-Wang,
HaiShaw,
ShangmingCai,
Ying1123,
hanming-lu,
huangtingwei9988,
hzh0425,
merrymercy,
xiezhq-hermann and
yizhang2077
as code owners
September 1, 2026 08:14
…/swa-drop-peer-mask # Conflicts: # python/sglang/srt/mem_cache/allocator/swa.py # test/registered/unit/mem_cache/test_swa_unittest.py
…to lsyin/swa-free-group
…/swa-drop-peer-mask
…to lsyin/swa-free-group
…/swa-drop-peer-mask
# Conflicts: # python/sglang/srt/mem_cache/allocator/swa.py # test/registered/unit/mem_cache/test_swa_unittest.py
Collaborator
Author
|
Latest PR Test (Base): ❌ Run #33573528101 PR chains passed |
ispobock
approved these changes
Sep 2, 2026
StevenChenSE
pushed a commit
to StevenChenSE/sglang
that referenced
this pull request
Sep 6, 2026
This was referenced Sep 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
free_swaonpage_size == 1gathers the peers and releases them without theswa_indices > 0filter; the filter's data-dependent output shape read a count back to the host on every SWA releasefree_fullfor released peers ([mem_cache] Addfree_kv_rowto release a request's kv row by row range #36721), grouped frees resolve peers at enqueue ([misc] Resolve SWA ownership at enqueue time for grouped free() #36646), and duplicate insert slices are split at the eviction floor ([mem_cache] Split duplicate insert frees at the SWA eviction floor #37481), so the gather never reads a padding entryswa.peer_mapped, an asyncInvariant(Bucket.FATAL_UNCONTAINABLE) checked at the gather underSGLANG_INVARIANT_CHECKfree_full:swa.peer_released(mapping[free_index] == 0,Bucket.GUARD) fires when a caller releases only the full side of a slot whose SWA peer is still live, which would otherwise leak that peer;free_group_endfrees the pending full ids directly so the check runs once, at enqueuepage_size > 1keeps one filter per group in_release_swa: HiCache LOAD_BACK re-pairs a page-aligned full chunk with an offset SWA one, so a page can hold unmapped slotsTests
TestSWAPeerMappedContract: the checked predicate is false exactly for a slot whose peer is gone, andfree_swano longer tripstorch.cuda.set_sync_debug_mode("error")(CUDA only, gated on the pre-fix form still being flagged)test_free_group_owns_tombstoned_indices:free()on a tombstoned slot is the caller error the contract now reportsfree_swa) before callingfree_full/ inserting withswa_evicted_seqlen, as window eviction does;test_free_full_leaves_the_mapping_intactis dropped because the mapping is always zero at that point under the contracttest_self_e2e_perturb_req_to_tokenruns withSGLANG_INVARIANT_CHECK=0: a perturbed slot reachingfree_swawith its peer released is the corruption under test, not a regressionFollows #36646.
CI States
Latest PR Test (Base): 🚫 Run #33678830613
Latest PR Test (Extra): 🚫 Run #33678830328
Latest PR Test (AMD ROCm 7.2): ⏳ Run #33678830257