Skip to content

[DSv4] Fix shared index cache accessor page size - #42832

Merged
kpham-sgl merged 4 commits into
sgl-project:mainfrom
kpham-sgl:codex/fix-dsv4-indexer-page-size
Oct 7, 2026
Merged

kpham-sgl merged 4 commits into
sgl-project:mainfrom
kpham-sgl:codex/fix-dsv4-indexer-page-size

Conversation

@kpham-sgl

@kpham-sgl kpham-sgl commented Oct 6, 2026 •

Copy link
Copy Markdown
Collaborator

DSv4's non-paged FP8 indexer prefill crashes because DeepSeekV4IndexerPool does not expose the storage page size expected by the shared DSA accessor. #42178 introduced this dependency; #42671 renamed the field to index_page_size. Reported in this comment.

Set index_page_size to the pool's physical page size.

Validation:

  • Pre-commit and git diff --check passed.
  • Manual H200 check using real DSv4 pools and the unmodified Triton gather: main raises the missing index_page_size error for 64- and 128-slot pages; the PR returns exactly the expected keys and scales for two sequences with non-sequential pages and partial final pages.
  • Manual DSv4 Flash FP8 server comparison on 4 H200s, TP4: a 9000-token prefill enters the non-paged gather on all four ranks with its default enablement and 8192-row threshold. Main crashes with the reported attribute error; the PR completes prefill and generates 8 tokens (HTTP 200).
  • Server comparison used a 16384-token prefill chunk, 32768-token cache budget, SWA/full ratio 1, and disabled graphs/radix caching to keep the full request eligible. DeepGEMM compiled on demand. This was a functional reproduction, not an accuracy or performance benchmark.

CI States

Latest PR Test (Base): ❌ Run #37577779907
Latest PR Test (Extra): ❌ Run #37577779610
Latest PR Test (AMD ROCm 10): ❌ Run #37577779910

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/unit/mem_cache/test_dsv4_compressed_pools.py test/registered/disaggregation/test_disaggregation_dsv4.py

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/mem_cache/test_dsv4_compressed_pools.py

🚀 8-gpu-h200 (1 test): ❌ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_dsv4.py

Copy link
Copy Markdown
Collaborator Author

Manually verified on 4×H200 (TP4) with DSv4 Flash FP8: a 9000-token prefill hits the non-paged path with its default 8192-row threshold. Main crashes with the missing index_page_size; this fix completes generation successfully. Real GPU gather checks also pass for 64- and 128-slot pages. Test changes have been reverted; the PR is now a one-line fix.

@kpham-sgl

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/disaggregation/test_disaggregation_dsv4.py

@github-actions

github-actions Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

🚀 8-gpu-h200 (1 test): ✅ View workflow run

cd test/ && python3 registered/disaggregation/test_disaggregation_dsv4.py

@kpham-sgl
kpham-sgl merged commit e230930 into sgl-project:main Oct 7, 2026
100 of 116 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant