[4/N][warmup][DSv4] Migrate common attention kernels by LopezCastroRoberto · Pull Request #50176 · vllm-project/vllm

LopezCastroRoberto · 2026-07-28T19:26:20Z

Depends on: #50175

For more details, see parent PR: #49627 and tracking issue #49349

Description

This PR migrates DSv4 common attention preparation, cache, indexer, and sparse-attention kernels to the shared warmup contract.

What Changed

Migrated fused Q/K normalization and MTP input normalization.
Migrated cache compression, inverse-RoPE quantization, and partial-state kernels.
Migrated common Triton indexer and sparse-attention kernels.
Defined kernel-owned compile keys and AST-traced warmup spaces.
Added compile-only paths without dummy kernel launches.
Added tests comparing dispatch results with pre-migration specialization logic.

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

mergify · 2026-07-29T10:37:38Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @LopezCastroRoberto.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

coderabbitai · 2026-09-04T15:06:02Z

⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

LopezCastroRoberto · 2026-09-07T11:21:00Z

@coderabbitai full review

coderabbitai · 2026-09-07T11:21:06Z

✅ Action performed

Full review finished.

Co-authored-by: Codex <codex@openai.com> Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>

LucasWilkinson

Overall LGTM, left a comment would be nice to address before landing

LucasWilkinson · 2026-09-07T19:18:59Z

I think we should not do warmup unit tests (especially specific ones like this) and instead just setting jit_monitor_mode to error for models we've scrubbed

LucasWilkinson · 2026-09-07T19:19:47Z

/ci run

github-actions · 2026-09-07T19:20:05Z

✅ Triggered Buildkite CI #87579 for commit 70b0b4218e72.

github-actions · 2026-09-07T19:20:35Z

✅ @LopezCastroRoberto, CI is now available for this PR.

/ci run starts upstream CI; /amd-ci run starts AMD CI only.
/ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
/amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
/ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>

LopezCastroRoberto · 2026-09-07T21:57:07Z

/ci run

github-actions · 2026-09-07T21:57:24Z

✅ Triggered Buildkite CI #87586 for commit 109045dae96d.

LopezCastroRoberto · 2026-09-07T22:53:40Z

/ci retry

github-actions · 2026-09-07T22:53:56Z

✅ The previous CI build is still running: https://buildkite.com/vllm/ci/builds/87586

LopezCastroRoberto · 2026-09-08T12:31:21Z

/ci run

github-actions · 2026-09-08T12:31:40Z

✅ Triggered Buildkite CI #87688 for commit 4c7c4fd5ed47.

Split out from #56214: include only the changes under vllm/models/. Adds the new vllm/models/deepseek_v4_1/ model definition package (nvidia, amd, and common code paths) and the associated vllm/models/deepseek_v4/ modifications. Rebased onto main: fused_inv_rope_fp8_quant.py was ported onto the VllmTritonJitKernel class structure from #50176, keeping the PDL warmup fix from #56035, while preserving this change's QUANTIZE toggle, NOPE_DIM-based rope offset, and vectorized UE8M0 scale packing. Co-authored-by: Kimi Code <noreply@moonshot.cn> Signed-off-by: Roger Wang <hey@rogerw.io>

) Signed-off-by: LopezCastroRoberto <rocastro@redhat.com> Co-authored-by: Codex <codex@openai.com> Signed-off-by: Jyotirmoy Roy <jyotirmoyroy649@gmail.com>

zyongye · 2026-09-14T19:22:51Z

@LopezCastroRoberto I just notice this change. One nit suggestion I have is that for triton kernel can we use the original kernel name instead of kernel? Otherwise on profiling trace is will only show kernel which shows no information. Cutedsl kernel is ok.

LopezCastroRoberto requested review from AndreasKaratzas, DarkLight1337, ProExpertProg, WoosukKwon, houseroad, mgoin, pavanimajety, robertgshaw2-redhat, tlrmchlsmth, yewentao256, youkaichao, ywang96 and zyongye as code owners July 28, 2026 19:26

claude Bot reviewed Jul 28, 2026

View reviewed changes

LopezCastroRoberto changed the title ~~[1/N][warmup][DSv4] Migrate attention kernels~~ [2/N][warmup][DSv4] Migrate attention kernels Jul 28, 2026

mergify Bot added deepseek Related to DeepSeek models v1 labels Jul 28, 2026

This was referenced Jul 28, 2026

[DO NOT MERGE][Perf][Feat] DSv4 de-JITification #49627

Open

Zero JIT compilation during runtime #49349

Open

mergify Bot added the needs-rebase label Jul 29, 2026

LopezCastroRoberto marked this pull request as draft July 29, 2026 17:11

LopezCastroRoberto mentioned this pull request Aug 12, 2026

[3/N][Feat][Perf] Add new warmup infrastructure for JITs. Add provider registry and orchestration for JIT warmup #50174

Merged

mergify Bot added inkling DSv4 labels Aug 19, 2026

LopezCastroRoberto force-pushed the warmup_migration/dsv4_3 branch from bda1ea1 to f767dbf Compare August 24, 2026 10:17

mergify Bot added kimi k3 and removed needs-rebase labels Aug 24, 2026

LopezCastroRoberto changed the title ~~[2/N][warmup][DSv4] Migrate attention kernels~~ [4/N][warmup][DSv4] Migrate common attention kernels Aug 24, 2026

mergify Bot removed the needs-rebase label Sep 4, 2026

LucasWilkinson reviewed Sep 4, 2026

View reviewed changes

LopezCastroRoberto requested a review from LucasWilkinson September 7, 2026 11:09

[4/N][warmup][DSv4] Migrate common attention kernels

70b0b42

Co-authored-by: Codex <codex@openai.com> Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>

LopezCastroRoberto force-pushed the warmup_migration/dsv4_3 branch from 961a539 to 70b0b42 Compare September 7, 2026 15:37

LucasWilkinson approved these changes Sep 7, 2026

View reviewed changes

LucasWilkinson added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 7, 2026

remove unit test

109045d

Signed-off-by: LopezCastroRoberto <rocastro@redhat.com>

Merge branch 'main' into warmup_migration/dsv4_3

4917e06

Merge branch 'main' into warmup_migration/dsv4_3

4c7c4fd

LucasWilkinson merged commit 6ddbab0 into vllm-project:main Sep 8, 2026
91 checks passed

This was referenced Sep 9, 2026

[Bug][ROCm][DSv4] JIT warmup compiles launch_pdl=True Triton kernels and fails with invalid griddepcontrol on AMD #56030

Closed

[Bugfix][ROCm][DSv4] Skip launch_pdl=True JIT warmup when PDL is unsupported #56035

Merged

tuukkjs mentioned this pull request Sep 9, 2026

[Bugfix][ROCm] Avoid unsupported PDL warmup for fused QKV RMSNorm #56048

Closed

ywang96 mentioned this pull request Sep 10, 2026

[Model] DeepSeek-V4.1-Flash Model Definitions #56228

Merged

Conversation

LopezCastroRoberto commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What Changed

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

mergify Bot commented Jul 29, 2026

Uh oh!

coderabbitai Bot commented Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

LopezCastroRoberto commented Sep 7, 2026

Uh oh!

coderabbitai Bot commented Sep 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LucasWilkinson left a comment

Choose a reason for hiding this comment

Uh oh!

LucasWilkinson Sep 7, 2026

Choose a reason for hiding this comment

Uh oh!

LucasWilkinson commented Sep 7, 2026

Uh oh!

github-actions Bot commented Sep 7, 2026

Uh oh!

github-actions Bot commented Sep 7, 2026

Uh oh!

LopezCastroRoberto commented Sep 7, 2026

Uh oh!

github-actions Bot commented Sep 7, 2026

Uh oh!

LopezCastroRoberto commented Sep 7, 2026

Uh oh!

github-actions Bot commented Sep 7, 2026

Uh oh!

LopezCastroRoberto commented Sep 8, 2026

Uh oh!

github-actions Bot commented Sep 8, 2026

Uh oh!

Uh oh!

zyongye commented Sep 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

LopezCastroRoberto commented Jul 28, 2026 •

edited

Loading

coderabbitai Bot commented Sep 4, 2026 •

edited

Loading

coderabbitai Bot commented Sep 7, 2026 •

edited

Loading