Add ability to replace oot ops when using lora#37181
Merged
yaochengji merged 1 commit intovllm-project:mainfrom Mar 17, 2026
Merged
Add ability to replace oot ops when using lora#37181yaochengji merged 1 commit intovllm-project:mainfrom
yaochengji merged 1 commit intovllm-project:mainfrom
Conversation
kyuyeunk
changed the title
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a mechanism to support LoRA with custom out-of-tree (OOT) operators by using a get_new_class_type helper for type checks. The approach is sound, but the implementation is incomplete as several type checks in vllm/lora/layers/column_parallel_linear.py were not updated. This will prevent LoRA from working with custom OOT layers for those specific layer types. I've added a detailed comment pointing out the missed locations.
kyuyeunk
force-pushed
the
oot_lora_replace
branch
2 times, most recently
from
8247ff1 to
fe5134d
Compare
kyuyeunk
force-pushed
the
oot_lora_replace
branch
from
fe5134d to
99de277
Compare
kyuyeunk
requested review from
LucasWilkinson and
MatthewBonanni
as code owners
kyuyeunk
force-pushed
the
oot_lora_replace
branch
from
99de277 to
3faf745
Compare
yaochengji
approved these changes
Mar 16, 2026
github-actions
bot
added
the
ready
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
auto-merge was automatically disabled
March 16, 2026 22:23
Head branch was pushed to by a user without write access
kyuyeunk
force-pushed
the
oot_lora_replace
branch
from
3faf745 to
1bbe84b
Compare
Contributor
Author
|
CI has passed. @yaochengji, auto-merge was disabled after i did rebase, can you trigger merge for this pr again? thanks! |
Lucaskabela
pushed a commit
to Lucaskabela/vllm
that referenced
this pull request
Mar 17, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
5 tasks
andylolu2
pushed a commit
to andylolu2/vllm
that referenced
this pull request
Mar 18, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
wendyliu235
pushed a commit
to wendyliu235/vllm-public
that referenced
this pull request
Mar 18, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
fxdawnn
pushed a commit
to fxdawnn/vllm
that referenced
this pull request
Mar 19, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
khairulkabir1661
pushed a commit
to khairulkabir1661/vllm
that referenced
this pull request
Mar 27, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
Monishver11
pushed a commit
to Monishver11/vllm
that referenced
this pull request
Mar 27, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com> Signed-off-by: Monishver Chandrasekaran <monishverchandrasekaran@gmail.com>
JiantaoXu
pushed a commit
to JiantaoXu/vllm
that referenced
this pull request
Mar 28, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
vrdn-23
pushed a commit
to vrdn-23/vllm
that referenced
this pull request
Mar 30, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com> Signed-off-by: Vinay Damodaran <vrdn@hey.com>
EricccYang
pushed a commit
to EricccYang/vllm
that referenced
this pull request
Apr 1, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com> Signed-off-by: EricccYang <yangyang4991@gmail.com>
liuchenbing2026
pushed a commit
to liuchenbing2026/vllm
that referenced
this pull request
Apr 4, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
big-yellow-duck
pushed a commit
to EmbeddedLLM/vllm
that referenced
this pull request
Apr 8, 2026
Signed-off-by: Kyuyeun Kim <kyuyeunk@google.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Add ability to support lora when using custom ops. Otherwise, custom ops will not be replaced with Lora layers.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.