[Config] [AMD][DSV4] [tune][gfx950] wo_b/wq_b a8w8 blockscale bpreshuffle configs

[karverma-amd]

Summary

Add MI355X (gfx950) tuned a8w8 block-scale bpreshuffle GEMM configs for two
DeepSeek-V4-Pro attention projections that had no gfx950 rows and were falling back
to aiter's default config:

projection	shape (N, K)
wo_b (o_proj up)	7168 × 1024
wq_b (q up-proj)	8192 × 1536

Tuned across the M ladder spanning concurrency 4–256 (decode M=1 … prefill M=16384).
The tuner selects tuned cktile at large M.

Kernel result (MI355X, gfx950)

shape	M	default	tuned	speedup
wo_b (7168×1024)	16384	224 µs	148 µs	~1.34×
wq_b (8192×1536)	16384	~277 µs	224 µs	~1.24×

End-to-end result (DeepSeek-V4-Pro, TP8, 128k ISL / 1k OSL)

Long-context prefill is where these projections run at large M most often (a 128k prompt
chunks into 16 prefill forwards at M=8192). TTFT, 3 reps, conc 8:

rep	default	tuned
1	1682.9 ms	1549.4 ms
2	1531.1 ms	1520.3 ms
3	1956.5 ms	1844.0 ms
mean	1724 ms	1638 ms (−5.0%)

Tuned is faster in every rep (TTFT and total throughput). Decode is unaffected
(decode M is tiny), and GSM8K accuracy is unchanged (0.948).

How the config was generated

# untuned shapes: wo_b (7168,1024) + wq_b (8192,1536) across M = 1,2,4,...,16384
python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py \
    --preshuffle --libtype all \
    -i aiter/configs/a8w8_blockscale_bpreshuffle_untuned_gemm.csv \
    -o aiter/configs/model_configs/dsv4_a8w8_blockscale_bpreshuffle_tuned_gemm.csv

How it was validated

Server (DeepSeek-V4-Pro, MI355X, TP8):

export SGLANG_USE_AITER=1 SGLANG_DEFAULT_THINKING=1 SGLANG_DSV4_REASONING_EFFORT=high
export SGLANG_USE_ROCM700A=0 SGLANG_HACK_FLASHMLA_BACKEND=unified_kv_triton
export AITER_BF16_FP8_MOE_BOUND=0 SGLANG_OPT_USE_AITER_BATCHED_GEMM=1
export SGLANG_OPT_UNIFIED_CACHE_FREE_OUT_OF_WINDOW_SLOTS=1 SGLANG_TIMEOUT_KEEP_ALIVE=900

python3 -m sglang.launch_server \
    --model-path deepseek-ai/DeepSeek-V4-Pro --tensor-parallel-size 8 \
    --attention-backend dsv4 --page-size 256 --swa-full-tokens-ratio 0.10 \
    --kv-cache-dtype fp8_e4m3 --enforce-shared-experts-fusion \
    --tool-call-parser deepseekv4 --reasoning-parser deepseek-v4 \
    --chunked-prefill-size 8192 --mem-fraction-static 0.89 \
    --max-running-requests 16 --cuda-graph-max-bs 16 --context-length 200000 \
    --speculative-algorithm EAGLE --speculative-num-steps 3 \
    --speculative-eagle-topk 1 --speculative-num-draft-tokens 4 --watchdog-timeout 3600

Accuracy — GSM8K (1319 questions, 1319 parallel):

python3 benchmark/gsm8k/bench_sglang.py --num-questions 1319 --parallel 1319 --port 8888
# -> Accuracy: 0.948  (unchanged vs default config)

Perf — 128k ISL / 1k OSL, streaming TTFT probe (conc 8, 3 reps):

# streaming client: NUM_PROMPTS concurrent prompts of ~INPUT_WORDS words
# (~1.06 tok/word => ~128k tokens), OUTPUT_LEN new tokens, reports TTFT/ITL/throughput
PORT=8888 NUM_PROMPTS=8 CONCURRENCY=8 OUTPUT_LEN=1024 INPUT_WORDS=120800 \
    python3 load_probe.py     # 3 reps; TTFT means in the table above

# equivalent with the stock sglang serving benchmark:
python3 -m sglang.bench_serving --backend sglang --port 8888 \
    --dataset-name random --random-input-len 131072 --random-output-len 1024 \
    --num-prompts 8 --max-concurrency 8

Notes / scope

• gfx950 only; other architectures are unaffected (no rows added for them).
• Additive change to dsv4_a8w8_blockscale_bpreshuffle_tuned_gemm.csv only; no existing
  rows modified.
• The win is specific to large-M (prefill / long-context); it is intentionally
  neutral for small-M decode where these projections are a negligible fraction of the step.

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5279 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[coderfeli]

add 32k config or <32k will go into default