perf(kimi-k3): re-sweep MoE tactic tables for flashinfer 0.6.16, warn on stale tables

[dongjiyingdjy]

Summary

The flashinfer 0.6.16 pin bump (#869) silently orphaned the packaged Kimi-K3 MoE tactic tables: filenames embed the swept flashinfer version (0.6.16.dev20260730), the loader correctly refuses a version mismatch, and every K3 launch since has fallen back to the startup autotune window — whose native picks lose 16–83% on prefill-sized buckets. This PR re-sweeps both layout tables on 0.6.16 stable and makes the failure mode visible.

Changes

• Re-swept tables for ep=8,tp=1 and ep=1,tp=8 (B300, flashinfer 0.6.16, tune_max=8192, 21 buckets each). Tactic indices do not survive version changes — the trtllm-gen kernel enumeration reorders between builds, so 17/21 (ep=8) and 12/21 (tp=8) buckets map to different indices for the same tile families. Tables must be re-swept on every pin bump, never renamed.
• Stale-table warning in load_packaged_flashinfer_tuning_cache: a lookup miss that finds a same-model/layout/device table swept on a different flashinfer version now logs a WARNING naming the orphaned file and the re-sweep command (previously a generic INFO). The next pin bump that forgets the tables shows up in serving logs instead of being a silent perf regression.

Measured (B300 ×8, K3 w4a8, flashinfer 0.6.16)

Kernel level (moe_tactic_sweep, table tactic vs autotune-window native pick, cold-L2):

layout	bucket	table	native	native slower by
ep=8,tp=1	512	349 µs	636 µs	+82%
ep=8,tp=1	2048	390 µs	714 µs	+83%
ep=8,tp=1	8192	761 µs	902 µs	+19%
ep=1,tp=8	1024	409 µs	496 µs	+21%
ep=1,tp=8	4096	734 µs	893 µs	+22%
ep=1,tp=8	8192	962 µs	1118 µs	+16%

End-to-end A/B (same commit, same TP8 serve config, only variable = table file present; evalscope, 8×~4k-token prompts for prefill, 32-way × 256-token generations for decode):

metric	with table	without (autotune)	delta
prefill total throughput	13478 tok/s	12238 tok/s	+10.1%
prefill TTFT (mean)	1757 ms	2017 ms	−12.9%
decode output throughput	1163 tok/s	1152 tok/s	+0.9% (noise)
startup autotune window	13 s	5 min 31 s	25×

Decode parity is expected: decode buckets (≤32 rows) are where the native heuristic is already near-optimal, and attention/comm dilute the MoE share.

Both tables load-verified on current main (post-#820: table load seeds the startup window; seeded shapes are skipped at zero cost). Note the sweep is keyed to tune_max=8192 — serving with chunked_prefill_size > 8192 changes the bucket ladder and misses the table.

🤖 Generated with Claude Code