use skip_all_guards_unsafe to drop global_state and torch_function_mode_stack guards instead of previous hacks#36204
Conversation
Signed-off-by: Laith Sakka <lsakka@meta.com>
laithsakka
requested review from
BoyuanFeng,
ProExpertProg,
youkaichao and
zou3519
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request refactors the guard handling mechanism in torch.compile wrappers. It replaces a manual, hacky approach of monkey-patching torch._C._dynamo.guards.GuardManager to disable GLOBAL_STATE and TORCH_FUNCTION_MODE_STACK guards. The new implementation leverages the public torch.compiler.skip_all_guards_unsafe API, which is a cleaner and more robust way to achieve the same goal of dropping all guards during compilation. This change simplifies the _compilation_context and makes the code more maintainable and less dependent on internal PyTorch APIs. The changes look correct and are a good improvement.
zhxchen17
left a comment
zhxchen17
left a comment
There was a problem hiding this comment.
lgtm, cc @zou3519 @ProExpertProg
|
This PR is not ROCm compatible as ROCm is running on torch 2.9 version. The regression is addressed here: #37219 |
…de_stack guards instead of previous hacks (vllm-project#36204) Signed-off-by: Laith Sakka <lsakka@meta.com>
…de_stack guards instead of previous hacks (vllm-project#36204) Signed-off-by: Laith Sakka <lsakka@meta.com>
5 participants
Purpose
title.
Test Plan
tlparse with VLLM_USE_BYTECODE_HOOK=0
