[CI] Fix mypy for `vllm/attention` and `vllm/compilation` by hmellor · Pull Request #26482 · vllm-project/vllm

hmellor · 2025-10-09T11:49:06Z

This change brings us closer to not needing a custom mypy check. Part of #26533

This change brings us closer to not needing a custom `mypy` check. Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

vllm/attention/selector.py

vllm/config/vllm.py

gemini-code-assist

Code Review

This pull request aims to fix mypy issues to enable stricter type checking. While most changes correctly add or refine type hints, I've identified a few regressions. Specifically, a logic change in vllm/attention/selector.py removes the fallback to a default attention backend, which could break existing setups. Additionally, the logic for determining the default compilation level in vllm/config/vllm.py appears to be inverted with respect to the enforce_eager flag. Lastly, a type hint in vllm/v1/attention/backends/utils.py was changed incorrectly, which could lead to runtime errors. These issues are of high and critical severity and should be addressed.

vllm/config/vllm.py

vllm/v1/attention/backends/utils.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

vllm/config/device.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor · 2025-10-09T12:28:15Z

vllm/config/vllm.py

        else:
            assert self.compilation_config.level >= CompilationLevel.NO_COMPILATION
            assert self.compilation_config.level <= CompilationLevel.PIECEWISE
-            assert self.compilation_config.level <= 3


No longer needed because level is Field(default=None, ge=0, le=3) so Pydantic will catch it

vllm/config/compilation.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

mergify · 2025-10-09T12:46:02Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

zou3519 · 2025-10-21T14:21:37Z

vllm/config/vllm.py

+            if not (self.model_config is None or self.model_config.enforce_eager):
+                self.compilation_config.mode = CompilationMode.VLLM_COMPILE


The logic seems to have changed here. Did V0 deprecation proceed to a point that we can make these changes?

As far as I know, V0 is already gone

I think I've lived under a rock for too long, you're right - using the v0 engine just imports the v1 engine it looks like.

Some of the tests were relying on this behavior, but as long as they pass we're good

If any tests do rely on V0 defaults, I'll update them

mergify · 2025-10-23T00:20:04Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

…or/vllm into mypy-attention-compilation

mergify · 2025-11-11T16:57:43Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

yewentao256

Please fix the pre-commit and conflicts so that we can land this PR

ProExpertProg · 2025-12-01T19:46:24Z

@hmellor are you still planning to merge this PR?

hmellor · 2025-12-01T21:08:27Z

I merged from main locally a few days ago, but have been focusing on Transformers v5 the past couple of weeks.

Would you like to take over the compilation part?

hmellor · 2026-01-27T20:30:52Z

vllm/attention handled by fixed mypy warnings for files vllm/v1/attention with TEMPORARY workaround #31465
vllm/compilation handled by:

Fix mypy for vllm/attention and vllm/compilation

f823fa7

This change brings us closer to not needing a custom `mypy` check. Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor requested review from LucasWilkinson, ProExpertProg, WoosukKwon, houseroad, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners October 9, 2025 11:49

mergify bot added the v1 label Oct 9, 2025

Remove skipvalidation

ca03050

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

chatgpt-codex-connector bot reviewed Oct 9, 2025

View reviewed changes

vllm/attention/selector.py Outdated Show resolved Hide resolved

vllm/config/vllm.py Outdated Show resolved Hide resolved

gemini-code-assist bot reviewed Oct 9, 2025

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

vllm/v1/attention/backends/utils.py Outdated Show resolved Hide resolved

Review comments

b6d07fa

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor requested review from NickLucche, bigPYJ1151 and jikunshang as code owners October 9, 2025 12:08

mergify bot added the tpu Related to Google TPUs label Oct 9, 2025

hmellor added 2 commits October 9, 2025 14:11

Review comments 2

5e0d99e

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Ignore modules that only exist after install

27f8ad8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor commented Oct 9, 2025

View reviewed changes

vllm/config/device.py Outdated Show resolved Hide resolved

Remove unnecessary whitespace

1e59be7

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor commented Oct 9, 2025

View reviewed changes

vllm/config/compilation.py Outdated Show resolved Hide resolved

Remove now redundant cast

4574efc

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

mergify bot added the needs-rebase label Oct 9, 2025

hmellor enabled auto-merge (squash) October 17, 2025 08:52

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 17, 2025

hmellor added 7 commits October 17, 2025 12:42

torch.Device -> torch.types.Device

80bb357

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'main' into mypy-attention-compilation

2d88ded

Review comment aboud compile_sizes

b456ee8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Make it a before validator

60640a8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'main' into mypy-attention-compilation

c249ed2

Skip None validation for delayed defaults

e19b02c

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Fix type hint for get_kv_cache_spec

65b3d74

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

zou3519 reviewed Oct 21, 2025

View reviewed changes

zou3519 approved these changes Oct 21, 2025

View reviewed changes

Merge branch 'main' into mypy-attention-compilation

a511f41

mergify bot added the needs-rebase label Oct 23, 2025

hmellor added 2 commits October 23, 2025 11:11

Merge branch 'main' into mypy-attention-compilation

de444cc

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'mypy-attention-compilation' of https://github.com/hmell…

c76fc00

…or/vllm into mypy-attention-compilation

mergify bot removed the needs-rebase label Oct 23, 2025

mergify bot added the nvidia label Nov 11, 2025

github-project-automation bot added this to NVIDIA Nov 11, 2025

mergify bot added the needs-rebase label Nov 11, 2025

yewentao256 reviewed Nov 13, 2025

View reviewed changes

mergify bot added the cpu Related to CPU backends label Dec 17, 2025

hmellor closed this Jan 27, 2026

auto-merge was automatically disabled January 27, 2026 20:30
Pull request was closed

github-project-automation bot moved this to Done in NVIDIA Jan 27, 2026

hmellor deleted the mypy-attention-compilation branch January 27, 2026 21:15

		if not (self.model_config is None or self.model_config.enforce_eager):
		self.compilation_config.mode = CompilationMode.VLLM_COMPILE

Uh oh!

Conversation

hmellor commented Oct 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmellor Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Oct 9, 2025

Uh oh!

zou3519 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

hmellor Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

zou3519 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

hmellor Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Oct 23, 2025

Uh oh!

mergify bot commented Nov 11, 2025

Uh oh!

yewentao256 left a comment

Choose a reason for hiding this comment

Uh oh!

ProExpertProg commented Dec 1, 2025

Uh oh!

hmellor commented Dec 1, 2025

Uh oh!

hmellor commented Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hmellor commented Oct 9, 2025 •

edited by github-actions bot

Loading