[kevin-mii]

Part of the MiniMax-M3.1-Flash AMD series: #46693 (M3.1 call sites) is stacked on this PR and adds M3.1's rows to the folder this PR creates, with no code; #46694 (indexer CP) is stacked on #46693.

Motivation

A model on AMD often needs AITER tuned-config rows for shapes the pinned AITER doesn't have yet. DeepSeek-V4 carried its own installer for that (deepseek_common/amd/aiter_intree_tune.py), and MiniMax-M3.1 would have needed a second one. This PR makes it one shared mechanism: a new model adds a CSV and no code.

Modifications

• One folder of rows: python/sglang/srt/utils/aiter_tuned_configs/<table>/<model>.csv, each in the columns of one AITER table: a8w8_blockscale_bpreshuffle, a8w8_bpreshuffle, bf16 and fmoe.
• One installer: ModelRunner.load_model calls install_aiter_tuned_tables() (utils/aiter_tuned_tables.py) once, before the model looks up a GEMM or fused-MoE config, on ROCm with SGLANG_USE_AITER=1. For each table with rows it merges them into AITER's own resolution of the table and points the table's AITER_CONFIG_* at the result. A single path skips AITER's own merge, which refuses duplicate keys.
• Row semantics. Rows are keyed by exact shape, like AITER's model_configs/:
  • A row with base_* columns replaces AITER's row only while AITER still picks that base kernel. Once AITER merges or re-tunes the shape, AITER's row wins.
  • A row without them adds a shape AITER has no row for, and yields to any row AITER has.
  • A hipBLASLt row names its kernel by an index into one hipBLASLt build, so it carries the ROCm that numbered it (rocm column) and applies only there.
  • A FlyDSL row applies only on an AITER that can import FlyDSL.
  • A table whose AITER_CONFIG_* the user set is left alone.
• DeepSeek-V4: its 32 aiter#6004 / [Feature] EP support for AWQ or GPTQ quantization of MoE models #6029 rows move into the folder unchanged. Its module keeps only the Triton JSON ([Feature] Dynamic YaRN for Qwen3 models #6030) and mHC tile_m=16 patches. Behavior changes:
  • its rows now apply to any model with those shapes, not only DeepSeek-V4;
  • a user-set AITER_CONFIG_GEMM_A8W8_BLOCKSCALE_BPRESHUFFLE now overrides them.
  • On the pinned AITER, every ROCm server with SGLANG_USE_AITER=1 therefore writes one merged bpreshuffle table to $TMPDIR/sglang_aiter_intree_<uid>/ at startup and points AITER at it.
• DeepSeek-V4 mHC fix: the tile_m=16 wrapper passed w_preshuffle_bf16 / res_preshuffle to an AITER whose get_mhc_fused_post_pre_config(m, hidden_size) has no such arguments, and raised TypeError. It now passes them only when AITER's signature has them.
• SGLANG_AITER_INTREE_GEMM_TUNE=0 still turns all of this off.

Tests

test/registered/amd/test_aiter_intree_tune.py (stage-b 1-GPU small, AMD):

• One synthetic table: a row replaces AITER's only over its base kernel, an addition only fills a missing shape, a merged or re-tuned AITER row wins, and no key repeats.
• A hipBLASLt row is dropped on another ROCm (ROCm 10 rejects ROCm 7.2's indices), using its own small CSV.
• On the pinned AITER every in-tree row in the folder installs and resolves to its own kernel.
• The mHC wrapper works on an AITER without the preshuffled-layout arguments; DeepSeek-V4's Triton JSON selection and mHC policy keep their CPU tests; the GPU lookup and mHC bit-identity tests are dropped.

Open question

A hipBLASLt row matches the runtime's ROCm major.minor. That assumes solution indices don't change between patch releases of one ROCm; a row that does go stale fails at graph capture with HIPBLAS_STATUS_INVALID_VALUE, not silently. This PR ships no hipBLASLt rows; #46693 adds the first ones (ROCm 7.2).

Validation

MI350X, ROCm 7.2 image, AITER acf8fdf93, SGLANG_USE_AITER=1:

• test_aiter_intree_tune: 6 passed. With the ROCm filter removed, its test fails.
• install_aiter_tuned_tables() on this AITER applies all 32 DeepSeek-V4 rows. On SGLang's Dockerfile pin (285579b62) and on AITER main (2026-11-02) all 32 still apply: chore: upgrade cutlass 3.9.2 #6004 and [Feature] EP support for AWQ or GPTQ quantization of MoE models #6029 are open, so the TODOs stay.
• Not run: DeepSeek-V4 serving. Its effective table is the same rows as before; only the installer changed.

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ❌ Run #42125016016
Latest PR Test (Extra): ❌ Run #42125016005
Latest PR Test (AMD ROCm 10): ❌ Run #42125015690

Tip

Base CI only covers the core cases, and its workload may not exercise your change. Use /rerun-test to run the tests your change touches, or copy this prompt for an agent:

Find the registered tests this PR affects and run them with /rerun-test, following https://github.com/sgl-project/sglang/blob/main/.agents/skills/rerun-test/SKILL.md

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.