Skip to content

Add explicit pooling classes for the Transformers backend #25322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 28 commits into from
Sep 30, 2025
Merged

Add explicit pooling classes for the Transformers backend #25322

merged 28 commits into from
Sep 30, 2025

Conversation

@hmellor
Copy link
Member

@hmellor hmellor commented Sep 20, 2025 •
edited
Loading

This PR:

  • Replaces TransformersModel with dedicated pooling classes:
    • TransformersEmbeddingModel
    • TransformersForSequenceClassification
  • Fixes the handling of the multiple pooler modules (self.model.pooler and self.pooler should both exist)
  • Adds special case for positions passed to roberta models (thanks @Isotr0py!)

@mergify mergify bot added the new-model Requests to new models label Sep 20, 2025
@hmellor
Switch to using pooler classes
492f1d5 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix runner type import
25667bd 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix pooler class resolution
56191d9 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor mentioned this pull request Sep 24, 2025
Merged
4 tasks
@hmellor
Update tested cheeckpoints
cb9cdae 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Check using_transformers_backend agains the actual class from the r…
8bb99b4 
…egistry

Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Use Transformers backend for Roberta
7edc010 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Don't replace Roberta yet
29f577f 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix bad name
76a9040 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Allow setting dtype in init_params
1725c0f 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Bert/Roberta only weight mapping for now
66cf2df 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Don't load lm_head for pooling models
3399476 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Partially fix classifier
c4f356d 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
pre-commit
5538654 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor moved this to In Progress in Transformers backend Sep 24, 2025
Isotr0py and others added 5 commits September 26, 2025 00:41
@Isotr0py
fix roberta edge case
db42a67 
Signed-off-by: Isotr0py <[email protected]>
@hmellor
Move TransformersPoolingBase to its own file
343394d 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix registry
10ab82a 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Fix pooling class resolution
cf98214 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor
Skip another unwanted head
853fc9e 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor changed the title Add-new-encoder-models Add explicit pooling classes for the Transformers backend Sep 29, 2025
@hmellor
Don't use forward hook
c2cb9a7 
Signed-off-by: Harry Mellor <[email protected]>
@hmellor hmellor marked this pull request as ready for review September 29, 2025 15:11
Isotr0py
Isotr0py reviewed Sep 30, 2025
View reviewed changes
vllm/model_executor/models/registry.py
Comment on lines 306 to 310
"TransformersEmbeddingModel": ("transformers_pooling", "TransformersEmbeddingModel"), # noqa: E501
"TransformersForSequenceClassification": ("transformers_pooling", "TransformersForSequenceClassification"), # noqa: E501
"TransformersForReward": ("transformers_pooling", "TransformersForReward"),
"TransformersForCausalLM": ("transformers", "TransformersForCausalLM"),
"TransformersForMultimodalLM": ("transformers", "TransformersForMultimodalLM"), # noqa: E501
"TransformersForMultimodalLM": ("transformers", "TransformersForMultimodalLM"), # noqa: E501
Copy link
Member

@Isotr0py Isotr0py Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs to update tests/models/registry.py too.

tests/models/test_transformers.py
model_impl="transformers") as vllm_model:
model_config = vllm_model.llm.llm_engine.model_config
assert model_config.using_transformers_backend()
vllm_kwargs = dict(
Copy link
Member

@Isotr0py Isotr0py Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add more models to test explicit pooling classes?

@mergify
Copy link

mergify bot commented Sep 30, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Sep 30, 2025
@mergify mergify bot removed the needs-rebase label Sep 30, 2025
@Isotr0py Isotr0py added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 30, 2025
@hmellor hmellor merged commit a388252 into vllm-project:main Sep 30, 2025
49 checks passed
@hmellor hmellor deleted the add-new-encoder-models branch September 30, 2025 22:07
@github-project-automation github-project-automation bot moved this from In Progress to Done in Transformers backend Sep 30, 2025
pdasigi pushed a commit to pdasigi/vllm that referenced this pull request Oct 2, 2025
@hmellor @Isotr0py @pdasigi
Add explicit pooling classes for the Transformers backend (vllm-proje…
92e28e7 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
yewentao256 pushed a commit that referenced this pull request Oct 3, 2025
@hmellor @Isotr0py @yewentao256
Add explicit pooling classes for the Transformers backend (#25322)
b3e1846 
Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Signed-off-by: yewentao256 <[email protected]>
tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025
@hmellor @Isotr0py @tomeras91
Add explicit pooling classes for the Transformers backend (vllm-proje…
8e00d2e 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Signed-off-by: Tomer Asida <[email protected]>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025
@hmellor @Isotr0py @xuebwang-amd
Add explicit pooling classes for the Transformers backend (vllm-proje…
2422ebb 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Signed-off-by: xuebwang-amd <[email protected]>
lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025
@hmellor @Isotr0py @lywa1998
Add explicit pooling classes for the Transformers backend (vllm-proje…
0a8c880 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025
@hmellor @Isotr0py @alhridoy
Add explicit pooling classes for the Transformers backend (vllm-proje…
db3e262 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025
@hmellor @Isotr0py @xuebwang-amd
Add explicit pooling classes for the Transformers backend (vllm-proje…
8525c89 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Signed-off-by: xuebwang-amd <[email protected]>
rtourgeman pushed a commit to rtourgeman/vllm that referenced this pull request Nov 10, 2025
@hmellor @Isotr0py @rtourgeman
Add explicit pooling classes for the Transformers backend (vllm-proje…
6db2b2d 
…ct#25322)

Signed-off-by: Harry Mellor <[email protected]>
Signed-off-by: Isotr0py <[email protected]>
Co-authored-by: Isotr0py <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Isotr0py Isotr0py Isotr0py approved these changes

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

@WoosukKwon WoosukKwon Awaiting requested review from WoosukKwon WoosukKwon is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao youkaichao is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@mgoin mgoin Awaiting requested review from mgoin mgoin is a code owner

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth tlrmchlsmth is a code owner

@houseroad houseroad Awaiting requested review from houseroad houseroad is a code owner

@yewentao256 yewentao256 Awaiting requested review from yewentao256 yewentao256 is a code owner

@ProExpertProg ProExpertProg Awaiting requested review from ProExpertProg ProExpertProg is a code owner

@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

Assignees

No one assigned

Labels

new-model Requests to new models ready ONLY add when PR is ready to merge/full CI is needed

Projects

Transformers backend
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@hmellor @Isotr0py