[FlyDSL] Add FP8 LiteTopK prefill operator

[AMD-yanfeiwang]

Summary

Add a gfx950 FP8 LiteTopK path for the released GLM-5.2-FP8 indexer contract: H=32, D=128, K=2048, measured with the production Q=8192 workload.

This is a single FP8 commit stacked on #5309. Q=1024, K=2048 is a valid operator shape when each row has at least 2048 candidates, but it is not the GLM reference workload and is not used for the table below.

Implementation

• Use native FP8 MFMA scoring over the production preshuffled page-64 cache ABI.
• Fuse seed calibration, suffix scoring, candidate filtering, stable exact TopK, and page mapping.
• Qualify K=2048, context through 1M, and a 16,384-candidate cap.
• Fail closed on overflow, underfill, invalid metadata/pages, nonfinite scores, and selector failures.

Validation

MI355X (gfx950), ROCm 7.2.4. These are operator benchmarks over deterministic tensors with the released GLM-5.2-FP8 shape, not a full-model E2E run. Each point is the median of three fresh processes; each process uses 5 warmup and 15 measured alternating dense/LiteTopK pairs. The reference column is the same Q8192/H32/D128/K2048 shape on B200 with captured GLM tensors from vLLM #48726; compare speedups, not absolute latency across hardware.

Context	MI355X dense	MI355X LiteTopK	MI355X	SGLang #32094	vLLM #48726	Recall
262,144	21.9057 ms	18.2898 ms	1.1977x	1.09x	1.2266x	99.999988%
524,288	47.2308 ms	36.3357 ms	1.2998x	1.17x	1.2652x	99.999964%
786,432	74.0062 ms	58.7514 ms	1.2597x	n/a	1.3084x	99.999964%
1,048,576	105.5670 ms	93.2930 ms	1.1316x	1.25x	1.2929x	99.999934%

Same-FlyDSL-backend exact-value validation passes for every row. Candidate maxima are 12,461, 14,012, 14,911, and 15,528, below the 16,384 cap.

Its author later reported a GLM-5.2 TP8+EP8 service result in SGLang #32094 of 134.717 s -> 86.004 s (1.566x), but that comment omits the context and full launch configuration, so it is reference-only rather than directly comparable.

• FP8 suite: 21 passed, 1 skipped; combined FP8/FP4 and two-GPU device tests pass.
• Black, Ruff, and git diff --check: pass.

AITER_USE_SYSTEM_TRITON=1 pytest -q op_tests/test_flydsl_pa_mqa_litetopk_fp8.py
AITER_USE_SYSTEM_TRITON=1 python op_tests/op_benchmarks/bench_flydsl_pa_mqa_fp8_litetopk.py \
  --model glm-5.2 --query-rows 8192 --raw-context 1048576 --warmup 5 --iterations 15

References

• FP4 base: [FlyDSL] Add FP4 LiteTopK prefill operator #5309
• Original implementation: https://github.com/Heisenberg-Yin/LiteTopK

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5348 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.