Skip to content

[Perf] dsv41: Use the fused ratio-2 compressor in the prefill graph - #46230

Merged
hnyls2002 merged 3 commits into
mainfrom
lsyin/dsv41-graph-fused-compress
Oct 24, 2026
Merged

hnyls2002 merged 3 commits into
mainfrom
lsyin/dsv41-graph-fused-compress

Conversation

@hnyls2002

@hnyls2002 hnyls2002 commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Inside the DeepSeek-V4.1 prefill CUDA graph, the ratio-2 compressor ran its torch path (dozens of small launches per source layer, plus a separate index-K store), because the fused compressor reads per-request extend offsets that were not graph-static. Eager extend already uses the fused kernel.

  • The prefill graph's metadata carries low_ratio_extend_offsets, an int32 [2, request slots] tensor of each request's first row and row count, refreshed in place at replay. Slots past the batch hold no rows, and DP padding rows belong to no request.
  • When the compressor uses the fused gate, the captured source projection writes | kv | score | rows into one buffer, and _low_ratio_compress_fused takes that projection (projected=) with the static offsets.
  • Compressors without the fused gate keep the torch path and its separate kv and score buffers, unchanged.

Verification

  • Existing DSv4 backend unit tests pass.

CI States

Latest PR Test (Base): 🚫 Run #40059037613
Latest PR Test (Extra): ❌ Run #40059037493
Latest PR Test (AMD ROCm 10): 🚫 Run #40059037529

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/rerun-test test_deepseek_v4.py test_c2_prefill_ring.py test_encoder_swa_replay.py test_dsv41_late_layer_tail_readers.py test_deepseek_v41_flash_gb300_encoder_bcg.py test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

@hnyls2002

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

🚀 4-gpu-b200 (2 tests): ✅ View workflow run

cd test/ && python3 registered/attention/unittests/dsv4/test_deepseek_v4.py
cd test/ && python3 registered/kernels/ops/attention/dsv4/test_c2_prefill_ring.py

🚀 1-gpu-h100 (1 test): ❌ View workflow run

cd test/ && python3 registered/attention/unittests/dsv4/test_deepseek_v4.py

🚀 ubuntu-latest (2 tests): ✅ View workflow run

cd test/ && python3 registered/unit/model_executor/test_encoder_swa_replay.py
cd test/ && python3 registered/unit/layers/test_dsv41_late_layer_tail_readers.py

🚀 2-gpu-gb300 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_encoder_bcg.py
cd test/ && python3 registered/e2e/models/test_deepseek_v41_flash_gb300_trtllm_bounded_bcg.py

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Oct 24, 2026
@hnyls2002
hnyls2002 merged commit 7d99630 into main Oct 24, 2026
235 of 293 checks passed
@hnyls2002
hnyls2002 deleted the lsyin/dsv41-graph-fused-compress branch October 24, 2026 04:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepseek run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant