Repository navigation
Conversation
kpham-sgl
requested review from
Fridge003,
HaiShaw,
Qiaolin-Yu,
hanming-lu,
hebiao064,
ispobock,
merrymercy,
yizhang2077 and
yuan-luo
as code owners
October 9, 2026 00:20
Collaborator
Author
|
/rerun-test test/registered/e2e/models/test_kimi_k3_b300.py test/registered/e2e/models/test_kimi_k3_b300_low_latency.py test/registered/e2e/models/test_glm53_flash_b200.py test/registered/attention/test_kda_prefill_flashinfer.py test/registered/unit/layers/attention/test_linear_attn_config.py |
Contributor
|
🚀 🚀 🚀 |
4 of 5 tasks
Collaborator
|
cc @leejnau |
5 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow up to #41400: safe-gate KDA models now select FlashInfer prefill by default on SM100/SM103, including Kimi-K3 and GLM-5.3-Flash. The choice is resolved once per model runner through the existing linear-attention backend resolver.
Explicit prefill flags take precedence. Unbounded gates, other devices, deterministic inference, two-batch overlap, full prefill CUDA graphs, and explicitly selected non-Triton base backends keep their existing selection. Decode and verify selection are unchanged.
The CPU policy test covers model/hardware eligibility, operator overrides, and phase isolation. Its fixture publishes the resolved CUDA graph configuration, fixing the fixture failure from the earlier default-enablement attempt.
Validation: repository pre-commit checks, Python syntax checks, and
git diff --checkpassed. Registered CPU, FlashInfer KDA kernel, Kimi-K3 B300, and GLM-5.3-Flash B200 suites will be requested through/rerun-teston this PR; these CI results are pending.Prior opt-in validation and limitations are recorded in #41400 (comment). GLM accuracy/speed, Kimi Balanced/MegaMoE accuracy, and Kimi MMMU-Pro passed locally. The Kimi single-request speed gate failed for both prefill backends on the two-node GB300 setup, and the single-prompt acceptance difference remains unexplained. CI should validate the default on the registered B200/B300 recipes before merge.
CI States
Latest PR Test (Base): ❌ Run #37864294127
Latest PR Test (Extra): ❌ Run #37864293813
Latest PR Test (AMD ROCm 10): ❌ Run #37864294101