[ROCm] gfx1250 optimizations and model enablement by jpvillam-amd · Pull Request #58800 · vllm-project/vllm

jpvillam-amd · 2026-09-25T21:53:10Z

Purpose

Enables and optimizes gfx1250 (ROCm) for gpt-oss-120b, DeepSeek-R1-0528, DeepSeek-V4-Flash and MiniMax-M3 (MXFP4). Every change is limited to gfx1250; other ROCm and CUDA paths behave as
before.

MoE: new AITER Triton/Gluon W4A4 MXFP4 expert (aiter_mxfp4_w4a4_moe.py), with its own backend enum so the oracle picks it only on gfx1250 (MI300/MI355 defaults unchanged). gfx1250
MX-scale swizzle and weight prep for W4A16/W4A8, router-logit sanitization and an fp32 routing bias, and routing-replay plumbing.
Linear: AITER preshuffled block-FP8 GEMM, used when a tuned config exists.
Attention / sampling: gfx1250 paths in the AITER unified-attention backend and an AITER fused sampler (sampler.py, topk_topp_sampler.py).
Models: DeepSeek-V4 AMD fixes and gpt-oss MXFP4 stride fix. gpt-oss RMSNorm+pad fusion uses the existing fuse_act_padding pass,
not a model-specific op.
Docker: gfx1250 base/image updates (ROCm SDK, Triton and AITER pins).

Not a duplicate: no other open PR targets gfx1250 enablement for these models. Kimi-K3 gfx1250 support was split out to be upstreamed separately.

Test Plan

gsm8k (lm-eval) for all models along side the normal server benchmarks

Test Result

#58800 (comment)

mergify · 2026-09-25T22:23:58Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jpvillam-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

JadenMathias · 2026-09-30T02:36:11Z

gfx1250_perf1 @ `e8a04fc` — results -- 15/10/2026, 3:14 PM EDT

Machine: gfx1250, 1 GPU (GPU3, TP1)
Stack: rocm/vllm-dev:custom_455_wip__torch__1014_b335, ROCm SDK 10.1.0a20260822, torch 2.11.0, triton aedd1f61, aiter f85b24b, vllm 0.9.2rc2.dev15749+ge8a04fcbf -> Editable Rebuild from source

gsm8k (5-shot, limit 100)

Model	flexible-extract	strict-match
gptoss	0.92 ± 0.0273	0.14 ± 0.0349
dsr1	0.96 ± 0.0197	0.26 ± 0.0441
dsv4f	0.94 ± 0.0239	0.94 ± 0.0239
minimax	0.92 ± 0.0273	0.92 ± 0.0273

vllm bench serve (random 1k/1k, conc 64, 640 prompts, warmup 128 prompts)

Metric	gptoss	dsr1	dsv4f	minimax
Output tok/s	8210.53	656.94	928.22	1392.46
Total tok/s	16421.06	1313.88	1856.44	2784.93
Req/s	8.02	0.64	0.91	1.36
Mean TTFT (ms)	882.39	48443.60	5182.32	24233.97
Mean TPOT (ms)	6.94	47.78	63.94	21.18
Mean ITL (ms)	6.95	47.78	63.94	21.18

mergify · 2026-10-02T12:09:05Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jpvillam-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2026-10-02T19:31:36Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jpvillam-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

mergify · 2026-10-07T17:23:58Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jpvillam-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>

Signed-off-by: Juan Villamizar <jpvillam@amd.com>

jpvillam-amd · 2026-10-14T20:17:16Z

/ci run

github-actions · 2026-10-14T20:18:03Z

✅ Triggered Buildkite CI #94932 for commit 0b0ae5975b40.

jpvillam-amd · 2026-10-14T22:13:47Z

/ci run

github-actions · 2026-10-14T22:14:08Z

✅ Triggered Buildkite CI #94956 for commit 9fbb90eddec3.

jpvillam-amd · 2026-10-14T22:22:52Z

/ci help

github-actions · 2026-10-14T22:23:08Z

❌ @jpvillam-amd, that is not a recognized CI command, so no CI was started. Post a new comment containing only one of: /amd-ci cancel, /amd-ci retry, /amd-ci run, /amd-ci run --allow-stale, /amd-ci run all, /amd-ci run all --allow-stale, /amd-ci run nightly, /amd-ci run nightly --allow-stale, /ci cancel, /ci retry, /ci run, /ci run --allow-stale, /ci run all, /ci run all --allow-stale, /ci run nightly, /ci run nightly --allow-stale.

mergify · 2026-10-15T14:38:07Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jpvillam-amd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>

jpvillam-amd · 2026-10-15T15:09:30Z

/ci run

github-actions · 2026-10-15T15:09:54Z

✅ Triggered Buildkite CI #95159 for commit e8a04fcbff1c.

jpvillam-amd · 2026-10-15T19:09:46Z

/ci retry

github-actions · 2026-10-15T19:10:11Z

✅ Queued 3 failed job(s) for retry in Buildkite CI #95159.

dllehr-amd

Thanks for all of the work on this @jpvillam-amd @danichan-mkm @JadenMathias !

vllm-project#58800 moved the gfx1250 check of rocm_unquantized_gemm_impl into use_aiter_triton_gemm and dropped on_gfx1250 from the import at the top of the function. That import now keeps only on_gfx942 next to main's names, for the gfx942 limit of 4 rows for wvSplitK. Signed-off-by: Markus Hartikainen <markus.hartikainen@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>

Signed-off-by: jpvillam <juan.villamizar@amd.com> Signed-off-by: Jaden Mathias <jaden.mathias@amd.com> Signed-off-by: Juan Villamizar <jpvillam@amd.com> Signed-off-by: jpvillam <jpvillam@amd.com> Signed-off-by: Omar Muhammad <omar.muhammad@amd.com> Signed-off-by: Daniel <danichan@amd.com> Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com> Co-authored-by: Jaden Mathias <jaden.mathias@amd.com> Co-authored-by: jpvillam <jpvillam@amd.com> Co-authored-by: Omar Muhammad <omar.muhammad@amd.com> Co-authored-by: Daniel <danichan@amd.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Daniel Chan <Daniel.Chan@amd.com>

mergify Bot added ci/build deepseek Related to DeepSeek models quantization gpt-oss Related to GPT-OSS models minimax labels Sep 25, 2026

github-project-automation Bot added this to gpt-oss Issues & Enhancements Sep 25, 2026

mergify Bot added DSv4 rocm Related to AMD ROCm labels Sep 25, 2026

github-project-automation Bot added this to AMD Sep 25, 2026

github-project-automation Bot moved this to To Triage in gpt-oss Issues & Enhancements Sep 25, 2026

github-project-automation Bot moved this to Todo in AMD Sep 25, 2026

mergify Bot added the needs-rebase label Sep 25, 2026

JadenMathias force-pushed the gfx1250_perf1 branch from 5d62d1b to 1ce2206 Compare September 30, 2026 02:38

mergify Bot removed the needs-rebase label Sep 30, 2026

JadenMathias force-pushed the gfx1250_perf1 branch from 1ce2206 to 49ede7f Compare September 30, 2026 14:54

mergify Bot added needs-rebase and removed needs-rebase labels Oct 2, 2026

mergify Bot added the needs-rebase label Oct 2, 2026

JadenMathias force-pushed the gfx1250_perf1 branch from c838e19 to 1d804ee Compare October 6, 2026 15:22

mergify Bot removed the needs-rebase label Oct 6, 2026

mergify Bot added the needs-rebase label Oct 7, 2026

JadenMathias force-pushed the gfx1250_perf1 branch from 001fa47 to b5512e9 Compare October 7, 2026 19:42

mergify Bot removed the needs-rebase label Oct 7, 2026

jpvillam-amd marked this pull request as ready for review October 7, 2026 19:59

jpvillam-amd requested a review from tjtanaa as a code owner October 7, 2026 19:59

JadenMathias and others added 3 commits October 14, 2026 15:05

Remove from priority backend (#1374)

f66ef0b

Signed-off-by: Jaden Mathias <jaden.mathias@amd.com>

Review meeting comments

48d006a

Signed-off-by: Juan Villamizar <jpvillam@amd.com>

Merge branch 'main' into gfx1250_perf1

0b0ae59

Merge branch 'main' into gfx1250_perf1

9fbb90e

mergify Bot added the needs-rebase label Oct 15, 2026

Merge branch 'main' into gfx1250_perf1

e8a04fc

Signed-off-by: Juan Villamizar <100237675+jpvillam-amd@users.noreply.github.com>

mergify Bot removed the needs-rebase label Oct 15, 2026

jpvillam-amd requested a review from dllehr-amd October 15, 2026 20:12

dllehr-amd approved these changes Oct 15, 2026

View reviewed changes

github-project-automation Bot moved this from To Triage to Ready in gpt-oss Issues & Enhancements Oct 15, 2026

mgoin approved these changes Oct 15, 2026

View reviewed changes

vllm-bot merged commit e711723 into vllm-project:main Oct 15, 2026
303 of 305 checks passed

github-project-automation Bot moved this from Todo to Done in AMD Oct 15, 2026

github-project-automation Bot moved this from Ready to Done in gpt-oss Issues & Enhancements Oct 15, 2026

maeehart mentioned this pull request Oct 15, 2026

[ROCm][Perf][DSv4.1] Mono decode on MI300X and MI325X (gfx942) #60731

Merged

6 tasks done

khluu mentioned this pull request Oct 16, 2026

[Bugfix][MoE] Keep the gate's hash_indices_table out of RoutedExperts parameters #61930

Merged

5 of 6 tasks

Rohan138 mentioned this pull request Oct 20, 2026

[ROCm] Enable expert parallelism for AITER Triton W4A16 MoE #59255

Open

Conversation

jpvillam-amd commented Sep 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

mergify Bot commented Sep 25, 2026

Uh oh!

JadenMathias commented Sep 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

gfx1250_perf1 @ e8a04fc — results -- 15/10/2026, 3:14 PM EDT

gsm8k (5-shot, limit 100)

vllm bench serve (random 1k/1k, conc 64, 640 prompts, warmup 128 prompts)

Uh oh!

mergify Bot commented Oct 2, 2026

Uh oh!

mergify Bot commented Oct 2, 2026

Uh oh!

mergify Bot commented Oct 7, 2026

Uh oh!

jpvillam-amd commented Oct 14, 2026

Uh oh!

github-actions Bot commented Oct 14, 2026

Uh oh!

jpvillam-amd commented Oct 14, 2026

Uh oh!

github-actions Bot commented Oct 14, 2026

Uh oh!

jpvillam-amd commented Oct 14, 2026

Uh oh!

github-actions Bot commented Oct 14, 2026

Uh oh!

mergify Bot commented Oct 15, 2026

Uh oh!

jpvillam-amd commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

jpvillam-amd commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

dllehr-amd left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jpvillam-amd commented Sep 25, 2026 •

edited

Loading

JadenMathias commented Sep 30, 2026 •

edited

Loading

gfx1250_perf1 @ `e8a04fc` — results -- 15/10/2026, 3:14 PM EDT