Fix SWA ownership across grouped frees - #36381
Merged
hnyls2002 merged 1 commit intoAug 25, 2026
Merged
Conversation
YazhiGao
marked this pull request as ready for review
August 25, 2026 21:30
YazhiGao
requested review from
alphabetc1,
hnyls2002 and
ispobock
as code owners
August 25, 2026 21:30
Collaborator
|
/rerun-test test_swa_unittest.py test_swa_eviction_boundary.py test_swa_lock_release_lifecycle.py test_swa_radix_cache_kl.py test_unified_radix_cache_kl_swa.py test_streaming_session_swa.py |
Contributor
|
Results for 🚀 |
Contributor
|
Results for 🚀 🚀 🚀 |
hnyls2002
approved these changes
Aug 25, 2026
elmo2019
pushed a commit
to elmo2019/sglang
that referenced
this pull request
Aug 27, 2026
Conflict in mem_cache/allocator/swa.py: sgl-project#36381 ("Fix SWA ownership across grouped frees") and this branch both appended text at the same spot, right after free_swa()'s clear_full_to_swa_mapping() call -- sgl-project#36381 added the ownership-resolving deferral block that completes free_swa(), this branch added free_full_segment / free_swa_segment / free_swa_page_reps. Resolved by taking sgl-project#36381's block first (it belongs to free_swa()'s body) and this branch's three methods after it. Note git presents the hunk in the opposite order; applying it as-presented would end free_swa() at the mapping clear -- freeing nothing -- and orphan `self.swa_attn_allocator.free(swa_indices)` inside free_swa_page_reps() with swa_indices undefined. free_group_end() auto-merged to sgl-project#36381's version (swa_attn_allocator.free() instead of re-entering free_swa()). The other eight files auto-merged. The new release paths already satisfy sgl-project#36381's invariant: free_swa_page_reps() reads full_to_swa_index_mapping at call time and defers only resolved SWA indices, and SWATokenToKVPoolAllocator.free_group_begin() does not put the sub-allocators into a free group, so they release eagerly.
saturn-acc
pushed a commit
to saturn-acc/sglang
that referenced
this pull request
Aug 31, 2026
nzr-niu
pushed a commit
to nzr-niu/sglang
that referenced
this pull request
Sep 1, 2026
Leoyzen
pushed a commit
to Leoyzen/sglang
that referenced
this pull request
Sep 3, 2026
jakki-amd
pushed a commit
to jakki-amd/sglang
that referenced
this pull request
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Root cause
Grouped free_swa calls deferred full-token IDs instead of resolving their current SWA mappings. Cache reconciliation could remap the same full-token ID before the group flushed, causing the replacement SWA slot to be freed while the original slot leaked.
This change transfers ownership at enqueue time by resolving and clearing the mapping immediately, then queues the physical SWA slot for the batched allocator free.
Test plan
Related: #36135 addresses the same underlying grouped-free ownership race with a different snapshot-barrier strategy.
-Robot
CI States
Latest PR Test (Base): ❌ Run #32901316992
Latest PR Test (Extra): ❌ Run #32901316710
Latest PR Test (AMD ROCm 7.2): ❌ Run #32901316920