[Config] Restore accurate Kimi-K3 A4W4 FMoE decode tiles

[amd-mghanimi]

Summary

• Kimi-K3 A4W4 NIAH accuracy dropped after the kimik3_a4w4_tuned_fmoe.csv decode tiles switched from flydsl_moe1 (~0.1% tune err) to flydsl_mxmoe (~1.1–2.1% err, plus cosine ~0.011 on token=1/2). Those rows landed in [HIP] [CK] [FlyDSL] [Kernel] Extend MXFP4 GEMM1 replacement to A4W4 #4526 (fe4814c64).
• Overlaying the old moe1 table on the new vLLM image recovered NIAH 4k (limit=50, C=32): baseline 0.82 → 0.94–0.96. This PR restores only the 8 high-err expert-FMoE rows (token 1,2,4,8,16,32,64,8192) and leaves the remaining moe1 tiles and shared-expert rows unchanged.
• Prefill token=4096 was already moe1 and is untouched. Decode tiles are what NIAH hits.

Test plan

• Overlay this CSV on the 3df4ae1 vLLM image and rerun niah_single_2 at 4k (limit=50, C=32); expect ~0.92–0.96 vs 0.82 on the mxmoe table
• Confirm remaining shapes (token 3,128–4096,16384,32768 and inter_dim=3072 shared-expert rows) still select the current kernels
• Optional: full NIAH 64k/128k after a vLLM rebuild that picks up this CSV

Made with Cursor

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5756 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[Copilot]

Copilot review overview

🔵 Needs a closer look

The change alters runtime kernel selection for accuracy-critical decode shapes and needs the stated NIAH validation run to confirm the expected quality recovery.

Review effort: Lite
Findings: None

What changed in this PR

Restores previously accurate FlyDSL MoE decode-tile kernel selections for the Kimi-K3 A4W4 expert-FMoE configuration by reverting a subset of high-error rows from flydsl_mxmoe back to the older flydsl_moe1 choices.

Changes:

• Replaced flydsl_mxmoe stage1 kernel selections with flydsl_moe1 for the expert-FMoE decode token buckets 1, 2, 4, 8, 16, 32, 64, and 8192.
• Updated associated stage2 flydsl_moe2_layout_* kernels and timing/error metadata for those rows while leaving other rows unchanged.

File	Description
aiter/​configs/​model_configs/​kimik3_a4w4_tuned_fmoe.csv	Reverts 8 high-error decode rows back to moe1-based tiles to recover Kimi-K3 A4W4 NIAH accuracy.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.