[ROCm]Enable AITER MoE backend for MiniMax-M3-MXFP4

[qli88]

[ROCm][feature] Enable MiniMax-M3-MXFP4 with AITER MoE

This feature requires AITER version bump (latest version).

Accuracy test:

1. vLLM server start: VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_AITER_MOE=1 VLLM_USE_BREAKABLE_CUDAGRAPH=0 vllm serve /data/amd-MiniMax-M3-MXFP4/ --block-size 128 -tp 8 --attention-backend TRITON_ATTN --tool-call-parser minimax_m3 --enable-auto-tool-choice --reasoning-parser minimax_m3 --moe-backend aiter
2. gsm8k:
   lm_eval
   --model local-chat-completions
   --model_args "model=/data/amd-MiniMax-M3-MXFP4/,base_url=http://127.0.0.1:8000/v1/chat/completions,num_concurrent=32,max_gen_toks=16384"
   --tasks gsm8k
   --num_fewshot 5
   --batch_size 1
   --apply_chat_template
   --fewshot_as_multiturn

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
gsm8k	3	flexible-extract	5	exact_match	↑	0.9401	±	0.0065
		strict-match	5	exact_match	↑	0.9409	±	0.0065

[BowenBao]

why is this special case necessary for minimax?

[qli88]

This is to avoid 'pad-skipping' accuracy issue in this version of AITER. I will submit an issue on AITER side and after it is resolved, I will remove this.

[tjtanaa]

@qli88 hidden_pad is not used at all. Let's not include it. Just introduce intermediate_pad will do.

[Fangzhou-Ai]

Do we have any perf gain regarding this PR?

[Fangzhou-Ai]

Update my perf result on 8 MI355X
ISL/OSL 8k1k conc=4

============ Serving Benchmark Result ============
Successful requests:                     40        
Benchmark duration (s):                  86.71     
Total input tokens:                      294213    
Total generated tokens:                  36805     
Request throughput (req/s):              0.46      
Output token throughput (tok/s):         424.44    
Total Token throughput (tok/s):          3817.31   
---------------Time to First Token----------------
Mean TTFT (ms):                          274.54    
Median TTFT (ms):                        256.45    
P90 TTFT (ms):                           272.89    
P99 TTFT (ms):                           603.19    
P99.9 TTFT (ms):                         677.08    
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          8.92      
Median TPOT (ms):                        8.97      
P90 TPOT (ms):                           9.11      
P99 TPOT (ms):                           9.15      
P99.9 TPOT (ms):                         9.16      
---------------Inter-token Latency----------------
Mean ITL (ms):                           8.92      
Median ITL (ms):                         8.34      
P90 ITL (ms):                            8.39      
P99 ITL (ms):                            8.51      
P99.9 ITL (ms):                          226.08    
----------------End-to-end Latency----------------
Mean E2EL (ms):                          8468.78   
Median E2EL (ms):                        8354.85   
P90 E2EL (ms):                           9269.13   
P99 E2EL (ms):                           9645.25   
P99.9 E2EL (ms):                         9683.83   
==================================================

[Fangzhou-Ai]

This PR is NOT ready to merge @tjtanaa until aiter version is bumped to v0.1.15.post3

[tjtanaa]

We already have this padding and unpadding logic. How different is this padding frome the existing field?
@qli88

@BowenBao I remember AMD added this arguments for GPTOSS and is not used by other platform. So will need an explanation about the need to add more arguments.

CC @dllehr-amd

[BowenBao]

afaik the "xxx_unpadded" args was genuine and not GPTOSS only. Padding is essential to cover any model with shape not divisible by group size / tp size. It wasn't AMD specific as well.

That said this is good point, the pad value can be calculated from hidden_dim - hidden_dim_unpadded, so seems redundant.

[qli88]

This is to provide a model specific padding size to override "hidden_dim - hidden_dim_unpadded". This very case is to circumvent an AITER issue which will cause accuracy issue when performing pad-skipping calculations. I will create an issue on AITER side and will revisit this after that issue gets resolved.

[Rohan138]

FYI for NV the padding is handled by the FI kernels, see: #30647; AITER handles padding for some backends (FlyDSL and Triton) but not CK/CKTile, would be nice to have all of it handled by AITER and remove the AMD-specific padding bits from vLLM

[fxmarty-amd]

Along with EMULATION backend, the AITER MXFP4 MOE backend is the only correct one to run https://huggingface.co/amd/MiniMax-M3-MXFP4, so this PR is quite important.

The current backend TRITON_UNFUSED does not apply activation QDQ, so it does not really apply config.json that requests activation quantization. It could be supported in the future (maybe in #46491)

[tjtanaa]

This PR can only be merged after we upgrade AITER. current AITER v0.1.13.post1 does not have this import path. It will fail.

[tjtanaa]

LGTM. Only merged once aiter version is upgraded

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @qli88.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork