[dev] feat(moe): Add KDA Low-Rank Projections and Extend MLA Support - #6877

Open

yuzhongw-nvidia wants to merge 12 commits into

yuzhongw-nvidia:yuzhongw/kimi-k3-kda

Commits on Sep 4, 2026

feat(kda): add Kimi K3 low-rank projections
yuzhongw-nvidia
committed
7216dd3
fix(kda): preserve legacy fused projections
yuzhongw-nvidia
committed
daaad6c
test(kda): accept empty TE bias tensors
yuzhongw-nvidia
committed
4ead70a
fix(mla): fuse latent norms in hybrid specs
yuzhongw-nvidia
committed
471802c
Support nope for MLA.
yuzhongw-nvidia
committed
9fecc56
fix(mla): reject latent caching without RoPE
yuzhongw-nvidia
committed
3bea2f2
fix(mla): defer hybrid QK norm fusion to resolver
yuzhongw-nvidia
committed
8fab48e
Decouple GDN activation from FFN activation
yuzhongw-nvidia
committed
4480270
refactor(mla): remove unreachable cached-latent branch
yuzhongw-nvidia
committed
1c658da
Use TE checkpoint for quantized GDN recompute
yuzhongw-nvidia
committed
4d55e5c

Commits on Sep 7, 2026

test(kda): tighten projection spec coverage
yuzhongw-nvidia
committed
0ac2f30

Commits on Sep 9, 2026

fix(kda): communicate standalone projections directly
yuzhongw-nvidia
committed
e122279