Skip to content

Fix SWA eviction boundary and page-align chunked prefill#22470

Merged
hnyls2002 merged 10 commits intomainfrom
lsyin/swa-robustness
Apr 10, 2026
Merged

Fix SWA eviction boundary and page-align chunked prefill#22470
hnyls2002 merged 10 commits intomainfrom
lsyin/swa-robustness

Conversation

@hnyls2002
Copy link
Copy Markdown
Collaborator

@hnyls2002 hnyls2002 commented Apr 9, 2026

Summary

Fix SWA radix tree insert boundary bug when page_size > sliding_window_size:

  • _evict_swa could over-evict to exactly page_floor(seq_len), making all tokens being inserted fully evicted (case 3)
  • _insert_helper had no handling for case 3 (swa_evicted_seqlen == total_length), falling through to _add_new_node(swa_tombstone=False) for already-evicted tokens
  • This inflated swa_evictable_size_, caused negative usage, and led to potential double-free on subsequent tree eviction

Two-sided fix:

  • Preventive: subtract extra page_size in _evict_swa formula so the frontier never reaches the insert boundary
  • Defensive: add early return in _insert_helper for case 3 (free value, don't create node)

Test plan

  • /rerun-test test_swa_eviction_boundary.py
  • /rerun-test test_swa_unittest.py
  • /rerun-test test_swa_radix_cache_kl.py

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_swa_eviction_boundary.py

@github-actions
Copy link
Copy Markdown
Contributor

1-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/unit/mem_cache/test_swa_eviction_boundary.py

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_swa_eviction_boundary.py

@github-actions
Copy link
Copy Markdown
Contributor

1-gpu-h100 (1 test): View workflow run

cd test/ && python3 registered/unit/mem_cache/test_swa_eviction_boundary.py

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@hnyls2002
Copy link
Copy Markdown
Collaborator Author

/rerun-test test_swa_eviction_boundary.py test_swa_unittest.py test_swa_radix_cache_kl.py

@github-actions
Copy link
Copy Markdown
Contributor

1-gpu-h100 (3 tests): View workflow run

cd test/ && python3 registered/unit/mem_cache/test_swa_eviction_boundary.py
cd test/ && python3 registered/unit/mem_cache/test_swa_unittest.py
cd test/ && python3 registered/radix_cache/test_swa_radix_cache_kl.py

@hnyls2002 hnyls2002 merged commit 722e25a into main Apr 10, 2026
94 of 136 checks passed
@hnyls2002 hnyls2002 deleted the lsyin/swa-robustness branch April 10, 2026 05:09
pyc96 pushed a commit to pyc96/sglang that referenced this pull request Apr 14, 2026
yhyang201 pushed a commit to yhyang201/sglang that referenced this pull request Apr 22, 2026
@hnyls2002 hnyls2002 mentioned this pull request Apr 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants