chore: disable enable_cpp_symbolic_shape_guards #23048
Conversation
Signed-off-by: Xiao Liu <[email protected]>
xiszishu
requested review from
ProExpertProg,
youkaichao and
zou3519
as code owners
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request disables the enable_cpp_symbolic_shape_guards configuration in PyTorch Dynamo to improve compilation time. The change is implemented in a backward-compatible way, using a try-except block to handle different PyTorch versions gracefully. This ensures the code works correctly whether the configuration flag is present or not. The implementation is sound and I have no concerns.
| # Note: this config is not available in torch 2.6, we can skip | ||
| # if the config doesn't exist |
There was a problem hiding this comment.
Why are you using PyTorch 2.6?
There was a problem hiding this comment.
I guess my higher-level question is, why does vLLM CPU use PyTorch 2.6 instead of PyTorch 2.7?
There was a problem hiding this comment.
There is due to the issue with 2.7.0 on x86 cpu: #19258
|
@xiszishu are you able to merge main into this branch? |
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]> Signed-off-by: Duncan Moss <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]> Signed-off-by: Xiao Yu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
Signed-off-by: Xiao Liu <[email protected]>
|
what was the compile time win amount Contributor |
3 participants
Reimplementation of #20836, compatible with torch 2.6.0 now
Purpose
Disable enable_cpp_symbolic_shape_guards to optimize compute time while compatible with the older version pytorch (required by cpu only setup).
Test Plan
Test Result
(Optional) Documentation Update
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.