[compile] Turn standalone_compile back on#27460
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request aims to re-enable standalone_compile by default. However, the current change only updates the default value within a TYPE_CHECKING block in vllm/envs.py, which does not affect the runtime behavior. The runtime default for VLLM_USE_STANDALONE_COMPILE is still False. I've added a critical comment to point out the necessary change to make this effective, and also noted an outdated comment that should be updated.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
https://github.com/vllm-project/vllm/blob/57869e47444188488508c3c8730edca8f1af632a/vllm/envs.py#L496-L502
Setting VLLM_USE_STANDALONE_COMPILE to True inside the TYPE_CHECKING block only affects static type checking; it does not change runtime behavior. The actual value returned by envs.VLLM_USE_STANDALONE_COMPILE still comes from the environment_variables mapping where the default is "0", so the flag remains False unless explicitly set in the environment. Consequently make_compiler will continue to pick InductorAdaptor and standalone compile stays disabled despite the intent of the commit.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
https://github.com/vllm-project/vllm/blob/57869e47444188488508c3c8730edca8f1af632a/vllm/envs.py#L496-L502
Setting VLLM_USE_STANDALONE_COMPILE to True inside the TYPE_CHECKING block only affects static type checking; it does not change runtime behavior. The actual value returned by envs.VLLM_USE_STANDALONE_COMPILE still comes from the environment_variables mapping where the default is "0", so the flag remains False unless explicitly set in the environment. Consequently make_compiler will continue to pick InductorAdaptor and standalone compile stays disabled despite the intent of the commit.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
We turned it off due to it having a bad interaction with gemma3n. This has now been fixed in PyTorch 2.9. Test Plan: - `vllm serve google/gemma-3n-E2B-it -tp 1` Signed-off-by: Richard Zou <zou3519@gmail.com>
zou3519
force-pushed
the
standalone_compile_back
branch
from
57869e4 to
b4271a4
Compare
Signed-off-by: Richard Zou <zou3519@gmail.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Signed-off-by: Richard Zou <zou3519@gmail.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Signed-off-by: Richard Zou <zou3519@gmail.com>
Signed-off-by: Richard Zou <zou3519@gmail.com>
Signed-off-by: Richard Zou <zou3519@gmail.com>
2 participants
We turned it off due to it having a bad interaction with gemma3n. This has now been fixed in PyTorch 2.9.
Closes #24547
Test Plan:
vllm serve google/gemma-3n-E2B-it -tp 1