Skip to content

perf(flydsl): fuse Kimi-K3 KDA decode + f_b projection - #5

Closed
JohnQinAMD wants to merge 2 commits into
mainfrom
perf/kimi-k3-kda-fb-fusion
Closed

JohnQinAMD wants to merge 2 commits into
mainfrom
perf/kimi-k3-kda-fb-fusion

Conversation

@JohnQinAMD

Copy link
Copy Markdown
Owner

Summary

Two commits: (1) fuse the Kimi-K3 KDA decode path (causal conv + recurrent
update + gated RMSNorm) into one gfx950 kernel; (2) fold the 128x128 f_b_proj
into the KDA decode state core (head-local, in-workgroup).

Effect

  • Removes one low-occupancy GEMV + the g1 global materialization + a dependency
    bubble per KDA layer.
  • Complete-chain 1.36x (9.76 -> 7.15 us B1); banked endpoint 50.063 tok/s.

Provenance

Banked change #3 (see perf_changelog.md). Independent PR off main.

AI assistance (OpenAI Codex) was used; a human submitter must review every line
before merge.

Made with Cursor

@github-actions

Copy link
Copy Markdown

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:triton-300x Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5 --add-label <label>

@JohnQinAMD

Copy link
Copy Markdown
Owner Author

Superseded by #16, rebuilt as the same two reviewable decode-core and incremental f_b commits on current main. #16 resolves only the FlyDSL export drift, passes Ruff/diff checks, restores DCO/disclosure on both commits, and remains draft for fork Black/GPU CI.

@JohnQinAMD JohnQinAMD closed this Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant