Skip to content

[KDA] Default safe-gate Blackwell prefill to FlashInfer - #43246

Merged
kpham-sgl merged 1 commit into
sgl-project:mainfrom
kpham-sgl:codex/flashinfer-kda-prefill-default-ci
Oct 9, 2026
Merged

kpham-sgl merged 1 commit into
sgl-project:mainfrom
kpham-sgl:codex/flashinfer-kda-prefill-default-ci

Conversation

@kpham-sgl

@kpham-sgl kpham-sgl commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Follow up to #41400: safe-gate KDA models now select FlashInfer prefill by default on SM100/SM103, including Kimi-K3 and GLM-5.3-Flash. The choice is resolved once per model runner through the existing linear-attention backend resolver.

Explicit prefill flags take precedence. Unbounded gates, other devices, deterministic inference, two-batch overlap, full prefill CUDA graphs, and explicitly selected non-Triton base backends keep their existing selection. Decode and verify selection are unchanged.

The CPU policy test covers model/hardware eligibility, operator overrides, and phase isolation. Its fixture publishes the resolved CUDA graph configuration, fixing the fixture failure from the earlier default-enablement attempt.

Validation: repository pre-commit checks, Python syntax checks, and git diff --check passed. Registered CPU, FlashInfer KDA kernel, Kimi-K3 B300, and GLM-5.3-Flash B200 suites will be requested through /rerun-test on this PR; these CI results are pending.

Prior opt-in validation and limitations are recorded in #41400 (comment). GLM accuracy/speed, Kimi Balanced/MegaMoE accuracy, and Kimi MMMU-Pro passed locally. The Kimi single-request speed gate failed for both prefill backends on the two-node GB300 setup, and the single-prompt acceptance difference remains unexplained. CI should validate the default on the registered B200/B300 recipes before merge.


CI States

Latest PR Test (Base): ❌ Run #37864294127
Latest PR Test (Extra): ❌ Run #37864293813
Latest PR Test (AMD ROCm 10): ❌ Run #37864294101

@kpham-sgl

Copy link
Copy Markdown
Collaborator Author

/rerun-test test/registered/e2e/models/test_kimi_k3_b300.py test/registered/e2e/models/test_kimi_k3_b300_low_latency.py test/registered/e2e/models/test_glm53_flash_b200.py test/registered/attention/test_kda_prefill_flashinfer.py test/registered/unit/layers/attention/test_linear_attn_config.py

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 9, 2026
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

🚀 8-gpu-b300 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_kimi_k3_b300.py
cd test/ && python3 registered/e2e/models/test_kimi_k3_b300_low_latency.py

🚀 4-gpu-b200 (2 tests): ✅ View workflow run

cd test/ && python3 registered/e2e/models/test_glm53_flash_b200.py
cd test/ && python3 registered/attention/test_kda_prefill_flashinfer.py

🚀 ubuntu-latest (1 test): ✅ View workflow run

cd test/ && python3 registered/unit/layers/attention/test_linear_attn_config.py

@kpham-sgl
kpham-sgl merged commit 8e19db1 into sgl-project:main Oct 9, 2026
105 of 121 checks passed
@nvpohanh

Copy link
Copy Markdown
Collaborator

cc @leejnau

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants