[Bugfix] Fix for builtins (forward fix of pytorch/177558)#37234
[Bugfix] Fix for builtins (forward fix of pytorch/177558)#37234zou3519 merged 5 commits intovllm-project:mainfrom
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a monkey-patch to address an issue with serializing builtins in PyTorch's AOT compilation path, which is a forward-fix of an upstream PyTorch pull request. The change adds the patch in vllm/compilation/wrapper.py and updates the pre-commit checks to allow the necessary pickle import. The patch seems correct and is defensively implemented. My main feedback is on improving the maintainability of the patch by updating a placeholder in the docstring to reference the correct upstream pull request.
Lucaskabela
force-pushed
the
lucaskabela/monkeypatch_aot_compile_fix
branch
from
777c535 to
c540235
Compare
|
cc @zhxchen17 for forward fix here |
Lucaskabela
force-pushed
the
lucaskabela/monkeypatch_aot_compile_fix
branch
2 times, most recently
from
fec4c5a to
0317548
Compare
Lucaskabela
requested review from
BoyuanFeng,
ProExpertProg,
hmellor,
youkaichao and
zou3519
as code owners
Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
Lucaskabela
force-pushed
the
lucaskabela/monkeypatch_aot_compile_fix
branch
from
0317548 to
160e4da
Compare
| # =================================================== | ||
| # torch <2.12 GraphCaptureOutput.get_runtime_env monkeypatch | ||
| # =================================================== |
There was a problem hiding this comment.
do you have a test that would exercise this? Otherwise we may just silently break it in the 2.12 update
There was a problem hiding this comment.
This is in @hmellor's PR which surfaced the need for this monkeypatch
There was a problem hiding this comment.
tests/compile/fullgraph/test_multimodal_compile.py test with transformers backend fails without this PR's patch
There was a problem hiding this comment.
Looks like we have a bit of a stalemate, could we merge this first then I'll be using it in #30518?
|
cc @zhxchen17 too |
…ct#37234) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> Signed-off-by: neweyes <328719365@qq.com>
…ct#37234) Signed-off-by: Lucas Kabela <lucaskabela@meta.com> Signed-off-by: Rishi Puri <riship@nvidia.com>
…ct#37234) Signed-off-by: Lucas Kabela <lucaskabela@meta.com>
…xtral, MoE and Granite regressions (#1311) ## Summary This PR fixes a set of regressions introduced by recent upstream changes and observed in vLLM-Gaudi hourly validation. The branch now includes: - Pixtral HPUAttention projection path fix, - MoE dispatch and method override alignment updates for fused MoE and compressed tensors, - unit test updates to match the new MoE runner API usage, - fix hybrid model page size alignment for Granite 4.0-H. ## Related upstream PRs that introduced the regressions - vllm-project/vllm#37234 - vllm-project/vllm#35153 - vllm-project/vllm#36963 - vllm-project/vllm#38960 - vllm-project/vllm#35326 - vllm-project/vllm#37467 --------- Signed-off-by: Paweł Olejniczak <pawelx.olejniczak@intel.com>
4 participants
Purpose
Forward fix of pytorch/pytorch#177558
Test Plan
See #30518 - needed so that the updated unit test with transformers passes
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.