Repository navigation
[Refactor] dsv4: Keep only the two-level V4.1 indexer in the prefill graph - #45783
Merged
Merged
Conversation
… the captured full top-k and the torch prefill indexer switch
hnyls2002
requested review from
Fridge003,
HaiShaw,
Qiaolin-Yu,
hebiao064,
ispobock and
merrymercy
as code owners
October 22, 2026 08:49
Collaborator
Author
|
/rerun-test test_deepseek_v41_flash_gb300_tp2ep2.py test_deepseek_v41_flash_gb300_dp2ep2.py test_deepseek_v41_flash_gb300_encoder_bcg.py test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py test_deepseek_v4.py test_dsv41_prefill_sparse_indexer.py test_dsv41_late_layer_tail_readers.py test_encoder_swa_replay.py test_dsv41_candidate_blocks.py test_dsv4_nonpaged_indexer.py test_dsv41_candidate_graph_variants.py test_prefill_cuda_graph_runner.py |
Collaborator
Author
|
/tag-and-rerun-ci |
Contributor
|
🚀 🚀 🚀 🚀 |
…captured consumer its index page size; drop row_chunk and topk_transform_paged_torch
hnyls2002
requested review from
BBuf,
DarkSharpness,
HydraQYH,
celve and
yuan-luo
as code owners
October 22, 2026 18:41
hnyls2002
changed the title
At1a8
added a commit
to At1a8/sglang
that referenced
this pull request
Oct 23, 2026
Resolve the conflict with sgl-project#45783 in deepseek_v4_backend_hip_radix.py: main removed the SGLANG_DSV41_TORCH_PREFILL_INDEXER check from the extend branch of _low_ratio_index_topk and the PR removed the prefill CP assert next to it, so drop both.
zcnrex
added a commit
to zcnrex/sglang
that referenced
this pull request
Oct 23, 2026
Resolve conflicts with bf16 V4.1 index logits (sgl-project#44051, sgl-project#45880), grouped verify rows (sgl-project#45251), the two-level-only prefill graph (sgl-project#45783) and KV-shard index-K writes (sgl-project#44280): the adapter takes a logits_dtype and maps [batch, next_n] paged queries and schedules to grouped varlen rows on SM100. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SparseTableBackend), the path released checkpoints take on SM100:low_ratio_prefill_graphnow means exactly that and absorbs_prefill_graph_two_level. Any other config runs the ratio-1/2 sources as one eager break, as SM90 already does.FullTopKIndexer.topk_prefill_captured,topk_transform_paged_torch) with its candidate-window assert, and the c1/c2 paged indexer metadata every replay built for it (full-width page tables, row chunks, DeepGEMM schedules). The captured consumer read only the index page size from that metadata, nowCapturedPrefillInputs.index_page_size;refresh_for_breakable_cuda_graph_replay_dropstwo_level, andPagedIndexerMetadatadropsrow_chunk.SGLANG_DSV41_TORCH_PREFILL_INDEXER, a development switch that forced the torch prefill indexer on SM100; SM90 keeps the torch indexer. HIP stateslow_ratio_prefill_graph = Falseinstead of borrowing the CUDA property, which reads the CUDA backend's candidate indexer.No behavior change for released checkpoints: DeepSeek-V4.1-Flash and DeepSeek-V4.1-Flash-1017 set
candidate_topk_blocks: 2048, so SM100 already runs the two-level path and SM90 the eager break; a replay just no longer builds metadata that nothing reads. Supersedes #45073.CI States
Latest PR Test (Base): 🚫 Run #39834530155
Latest PR Test (Extra): ❌ Run #39834530688
Latest PR Test (AMD ROCm 10): ⏳ Run #39834530160