[AMD][DSv4] Add experimental AITER FP4 streaming TopK

[AMD-yanfeiwang]

Motivation

The HIP FP4 indexer currently materializes an FP32 [query_rows, c4_context] score matrix before selecting sparse KV positions. #37660 bounds allocator fragmentation with a device-global 2 GiB slab, and #38086 is an independent proposal that replaces it with an accounted 512 MiB workspace.

This draft takes the operator-level route instead: use the exact streaming score + TopK API proposed in ROCm/aiter#5282, so eager prefill does not produce the full logits matrix at all.

Changes

• Probe flydsl_pa_mqa_topk_fp4_prefill once during backend construction and freeze the selected execution path before graph capture.
• On supported gfx950 FP4 eager-prefill calls:
  • pass packed Q/K, weights and C4 page metadata directly to AITER;
  • write final physical C4 slots and optional raw indices in place;
  • skip SGLang's separate full-logits TopK path and never allocate the 2 GiB slab.
• Preserve deterministic compatibility fallback for:
  • installed AITER versions without the new symbol;
  • decode / CUDA-graph capture;
  • an explicit SGLANG_DSV4_FP4_FUSED_TOPK=false kill switch.
• Keep existing prefill metadata reuse for the guarded page table and identity row mapping. CP/TBO paths can build the bounded metadata inline and rely on stream-local allocator ordering; there is no shared logits buffer.
• Enable the FP4 indexer in the existing DeepSeek-V4 Flash/Pro MTP/CP/TBO end-to-end tests.
• Add capability-freeze and fused raw-to-physical output coverage.

No FP4LogitsWorkspace, ModelRunner workspace registry, KV-pool accounting field, or context-sized environment budget is added by this PR.

Dependency

Draft dependency: ROCm/aiter#5282.

The integration uses exact symbol probing rather than a package-version comparison, so existing ROCm images retain their current behavior. Once the AITER PR merges, this branch will pin its merge SHA in docker/rocm.Dockerfile and remove any superseded temporary AITER patch. Development/CI can test the dependency through AITER_COMMIT_OVERRIDE.

Memory geometry

At 768K full-token context, one C4 FP32 score row is approximately 0.75 MiB. A 1024-row local prefill therefore materializes roughly 768 MiB today. The dependent AITER path keeps bounded per-CTA candidates and merge scratch (about 20 MiB for rows=1024/topk=1024), independent of context length.

Validation

Completed:

• capability probe unit tests: 3 passed
• targeted pre-commit hooks: passed
• Python syntax/import checks: passed
• dependent AITER static and offline gfx950 compilation checks: passed

Queued on MI355X:

• AITER fused/tiled/full-logits runtime correctness for TopK 512/1024
• SGLang raw/physical index integration test
• long-context peak memory and microbenchmarks
• 256K/786K client-concurrency 128/256 end-to-end comparison against the legacy 2 GiB and [AMD][DSv4] Replace fixed FP4 indexer logits slab with adaptive workspace #38086 512 MiB paths

Graph-captured decode/verify remains on the existing path in this first draft. This PR remains draft until the dependency and GPU results are attached.

---

CI States

Latest PR Test (Base): ❌ Run #34035988444
Latest PR Test (Extra): ❌ Run #34035988299
Latest PR Test (AMD ROCm 7.2): ❌ Run #34035988501

[github-actions]

Thanks @AMD-yanfeiwang. Closing this because it is still a draft and has not been updated in 32 days.

Reopen it if the work is still relevant.

Some directories moved recently, so an older branch may need retargeting:
sgl-kernel/ -> python/sglang/kernels/aot/, python/sglang/jit_kernel/
-> python/sglang/kernels/jit/, docs/ -> docs/docs/ (.mdx),
bench_serving.py -> benchmark/serving.py, test/srt/ -> test/registered/.