Skip to content

[Fix] Write the req_to_token page tail so rows stay valid over whole pages - #35401

Open
hnyls2002 wants to merge 13 commits into
mainfrom
lsyin/page-tail-write
Open

hnyls2002 wants to merge 13 commits into
mainfrom
lsyin/page-tail-write

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Aug 18, 2026 •

Copy link
Copy Markdown
Collaborator

The paged allocator hands out whole pages, but the row write only covers the requested tokens, so req_to_token[:, seq_len : ceil(seq_len)] keeps the previous occupant's indices. Fill that tail (slots inside a page are consecutive) and widen the row headroom to page_size - 1 for every page_size > 1 config, not just speculative ones.


CI States

Latest PR Test (Base): ❌ Run #32304945912
Latest PR Test (Extra): ❌ Run #32304945576
Latest PR Test (AMD ROCm 10): ❌ Run #32304945813

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_page_tail_indices.py test_dllm_fdfo_kv_reuse.py test_unified_radix_cache_kl_full.py test_streaming_session_swa.py

@github-actions

github-actions Bot commented Aug 18, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test test_page_tail_indices.py test_dllm_fdfo_kv_reuse.py test_unified_radix_cache_kl_full.py test_streaming_session_swa.py:

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_page_tail_indices.py
cd test/ && python3 registered/unit/mem_cache/test_dllm_fdfo_kv_reuse.py

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/sessions/test_streaming_session_swa.py

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Aug 18, 2026
@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_page_tail_indices.py test_hisparse_allocator.py test_disaggregation_hisparse.py test_disaggregation_decode_radix_cache.py test/registered/disaggregation/test_disaggregation_basic.py

@github-actions

github-actions Bot commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test test_page_tail_indices.py test_hisparse_allocator.py test_disaggregation_hisparse.py test_disaggregation_decode_radix_cache.py test/registered/disaggregation/test_disaggregation_basic.py:

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_page_tail_indices.py
cd test/ && python3 registered/unit/mem_cache/test_hisparse_allocator.py

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_hisparse.py

🚀 8-gpu-h20 (1 test): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_decode_radix_cache.py

🚀 2-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_basic.py

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_page_tail_indices.py test_streaming_session_swa.py test_unified_radix_cache_kl_full.py test/registered/disaggregation/test_disaggregation_basic.py test_disaggregation_hisparse.py

@github-actions

github-actions Bot commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Results for /rerun-test test_page_tail_indices.py test_streaming_session_swa.py test_unified_radix_cache_kl_full.py test/registered/disaggregation/test_disaggregation_basic.py test_disaggregation_hisparse.py:

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_page_tail_indices.py

🚀 1-gpu-h100 (1 test): ✅ View workflow run

cd test/ && python3 registered/sessions/test_streaming_session_swa.py

🚀 2-gpu-h100 (2 tests): ✅ View workflow run

cd test/ && python3 registered/radix_cache/unified_radix_tree/test_unified_radix_cache_kl_full.py
cd test/ && python3 registered/disaggregation/test_disaggregation_basic.py

🚀 8-gpu-h200 (1 test): ❌ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_hisparse.py

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@Jiminator Jiminator closed this Sep 14, 2026
@Jiminator
Jiminator deleted the lsyin/page-tail-write branch September 14, 2026 04:44
@alexnails
alexnails restored the lsyin/page-tail-write branch September 14, 2026 05:43
@hnyls2002 hnyls2002 removed the run-ci CI: run the baseline test suite on this PR label Sep 14, 2026
@hnyls2002 hnyls2002 reopened this Sep 14, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants