Skip to content

[ROCm] Fix AttributeError for torch.compiler.skip_all_guards_unsafe on older PyTorch#37219

Merged
DarkLight1337 merged 3 commits intovllm-project:mainfrom
ROCm:akaratza_fix_compiler_wrapper
Mar 17, 2026
Merged

[ROCm] Fix AttributeError for torch.compiler.skip_all_guards_unsafe on older PyTorch#37219
DarkLight1337 merged 3 commits intovllm-project:mainfrom
ROCm:akaratza_fix_compiler_wrapper

Conversation

@AndreasKaratzas
Copy link
Collaborator

@AndreasKaratzas AndreasKaratzas commented Mar 16, 2026

Test plan

  • pytest -s-v tests/test_regression.py::test_max_tokens_none
  • pytest -s -v tests/v1/entrypoints/llm/test_struct_output_generate.py

cc @kenroche

…n older PyTorch

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly addresses a compatibility issue with older PyTorch versions by adding a fallback for torch.compiler.skip_all_guards_unsafe. I've suggested a small improvement to make the implementation more concise and Pythonic by using getattr, which enhances code maintainability.

Comment on lines +115 to +118
if hasattr(torch.compiler, "skip_all_guards_unsafe"):
options["guard_filter_fn"] = torch.compiler.skip_all_guards_unsafe
else:
options["guard_filter_fn"] = lambda x: [False for _ in x]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This if/else block can be simplified by using getattr with a default value. This approach is more concise and Pythonic for checking for an attribute and providing a fallback, which improves code readability and maintainability.

Suggested change
if hasattr(torch.compiler, "skip_all_guards_unsafe"):
options["guard_filter_fn"] = torch.compiler.skip_all_guards_unsafe
else:
options["guard_filter_fn"] = lambda x: [False for _ in x]
options["guard_filter_fn"] = getattr(
torch.compiler,
"skip_all_guards_unsafe",
lambda x: [False for _ in x])

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered getattr but went with hasattr instead. It makes the version-dependent branching more explicit and easier to spot. Added comments to clarify the intent behind each path.

…n older PyTorch

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
…n older PyTorch

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
@AndreasKaratzas AndreasKaratzas added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 16, 2026
Copy link

@gpolovets1 gpolovets1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @AndreasKaratzas thanks for this fix! Would you be able to expedite merging, as it is breaking some downstream tests for TPU as well.
Thanks!

@DarkLight1337 DarkLight1337 merged commit 54a62a7 into vllm-project:main Mar 17, 2026
56 checks passed
@github-project-automation github-project-automation bot moved this from Todo to Done in AMD Mar 17, 2026
@AndreasKaratzas AndreasKaratzas deleted the akaratza_fix_compiler_wrapper branch March 17, 2026 03:50
khluu pushed a commit that referenced this pull request Mar 17, 2026
…n older PyTorch (#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
(cherry picked from commit 54a62a7)
zhenwei-intel pushed a commit to zhenwei-intel/vllm that referenced this pull request Mar 17, 2026
…n older PyTorch (vllm-project#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Lucaskabela pushed a commit to Lucaskabela/vllm that referenced this pull request Mar 17, 2026
…n older PyTorch (vllm-project#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
andylolu2 pushed a commit to andylolu2/vllm that referenced this pull request Mar 18, 2026
…n older PyTorch (vllm-project#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
wendyliu235 pushed a commit to wendyliu235/vllm-public that referenced this pull request Mar 18, 2026
…n older PyTorch (vllm-project#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
fxdawnn pushed a commit to fxdawnn/vllm that referenced this pull request Mar 19, 2026
…n older PyTorch (vllm-project#37219)

Signed-off-by: Andreas Karatzas <akaratza@amd.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed rocm Related to AMD ROCm

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants