perf(flydsl): fuse Kimi-K3 KDA decode and f_b projection - #16
Open
JohnQinAMD wants to merge 6 commits into
Open
JohnQinAMD wants to merge 6 commits into
JohnQinAMD wants to merge 6 commits into
Conversation
Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com> Assisted-by: OpenAI Codex
Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com> Assisted-by: OpenAI Codex
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
Sort public imports and exports, use functools.cache, and annotate intentional FlyDSL stream defaults for the repository-wide lint configuration. Assisted-by: OpenAI Codex Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com>
JohnQinAMD
marked this pull request as ready for review
July 31, 2026 21:36
Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com>
Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com>
JohnQinAMD
force-pushed
the
perf/kimi-k3-kda-fb-clean
branch
from
August 2, 2026 02:08
37943cc to
762b3e9
Compare
op_tests/op_benchmarks/flydsl/ does not exist on main, no CI job references it, and none of the 84 existing flydsl kernel modules ship a companion benchmark. The script is kept out of tree; measured numbers go in the PR description instead. Signed-off-by: Yanyuan Qin <yanyuan.qin@amd.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a gfx950 Kimi-K3 KDA decode kernel, then incrementally fuse the head-local
128x128
f_bprojection into the same workgroup.The first commit fuses causal convolution, recurrent state update, and gated
RMSNorm. The second removes the low-occupancy
f_bGEMV, its globalg1materialization, and the dependency bubble before the state core.
Dispatch and fallback
The public wrapper validates device, shape, dtype, stride, cache layout, batch,
and gate-bound contracts. Unsupported requests are rejected before launch and
remain owned by the vLLM fallback. The kernel is exposed only through the
FlyDSL availability boundary.
Performance
Kimi-K3 TP8 batch-one decode on MI355X:
f_bThe accepted endpoint moved from 47.644 to 50.063 tok/s (+5.1%) and reached
19.953 ms TPOT.
Correctness and validation
and fused-vs-unfused output/state comparison.
all 9 focused cases in the immutable MI355X validation image, including both
canonical convolution-cache layouts.
GSM8K first 100 at 100/100 with zero invalid responses.
git diff --checkpasses.Fork Black and Ruff CI pass. The PR is ready for review; hardware workflows
were triggered by the ready-for-review transition.
Disclosure
Developed with assistance from OpenAI Codex. The author reviewed the kernel
contracts, numerical path, tests, and benchmark evidence.