Improve Transformers v4/v5 compatibility in tokenizers and processors by cccat6 · Pull Request #34768 · vllm-project/vllm

cccat6 · 2026-02-18T02:27:38Z

Summary

Improve tokenizer compatibility across Transformers v4/v5 without global monkey patches.
Add processor compatibility retry for remote processors that still pass optional_attributes.
Make Ovis2.5 special-token handling robust across Transformers v4/v5.
Support huggingface-hub v1 transfer flags while preserving older hub behavior.
Update targeted multimodal test input normalization for GLM-ASR minimum audio length.

Validation

transformers==4.56.2
- pytest -q -rs tests/model_executor/test_weight_utils.py
- pytest -q -rs tests/models/multimodal/processing/test_common.py::test_processing_correctness -k 'AIDC-AI/Ovis2.5-2B or allenai/Molmo2-8B or allenai/Molmo2-O-7B or zai-org/GLM-ASR-Nano-2512'
transformers==4.57.4
- Same commands, pass.
transformers==5.1.0
- Same commands, pass.

Transformers 5.2.0 validation

Updated dependency cap: transformers >= 4.56.0, <= 5.2.0.
Re-tested locally in a fresh GPU-enabled environment with transformers==5.2.0:
- pytest -q -rs tests/model_executor/test_weight_utils.py (2 passed)
- pytest -q -rs tests/models/multimodal/processing/test_common.py::test_processing_correctness -k 'AIDC-AI/Ovis2.5-2B or allenai/Molmo2-8B or allenai/Molmo2-O-7B or zai-org/GLM-ASR-Nano-2512' (12 passed, 351 deselected)

gemini-code-assist

Code Review

This pull request introduces several improvements to enhance compatibility with different versions of transformers and huggingface-hub.

The enable_hf_transfer function is now compatible with both old and new APIs for transfer acceleration.
A robust compatibility layer has been added to handle older custom processors and tokenizers with newer versions of the transformers library, using temporary patches instead of global monkey-patching.
Special token handling for Ovis2.5 has been improved to be more robust.
Tests have been updated to be more comprehensive and to properly clean up global state modifications.

The changes are well-implemented and address important compatibility issues. The code quality is high, and the solutions are pragmatic and well-designed. I have no major concerns and approve of these changes.

mergify · 2026-02-18T02:32:15Z

Hi @cccat6, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

Signed-off-by: Codex Bot <codex-bot@users.noreply.github.com>

cccat6 · 2026-02-18T02:42:26Z

Addressed CI feedback in force-pushed commit 54cc897e2: ran pre-commit fixes (ruff check/format) and added DCO sign-off.

gaby · 2026-02-18T05:17:47Z

@cccat6 Can you also test transformers 5.2

https://github.com/huggingface/transformers/releases/tag/v5.2.0

cccat6 · 2026-02-18T07:17:41Z

@cccat6 Can you also test transformers 5.2

https://github.com/huggingface/transformers/releases/tag/v5.2.0

Will do. Thanks.

cccat6 · 2026-02-18T07:27:04Z

Follow-up on compatibility testing request:

I created a fresh local venv (/workspace/.venv-tf53) and re-ran targeted regression in that isolated environment.

Validated with transformers==5.2.0:

pytest -q -rs tests/model_executor/test_weight_utils.py -> 2 passed
pytest -q -rs tests/models/multimodal/processing/test_common.py::test_processing_correctness -k 'AIDC-AI/Ovis2.5-2B or allenai/Molmo2-8B or allenai/Molmo2-O-7B or zai-org/GLM-ASR-Nano-2512' -> 12 passed

No additional code changes were required from this round.

cccat6 · 2026-02-18T07:50:12Z

Update: addressed the version-cap request and validated on Transformers 5.2.0.

Changed requirements/common.txt to transformers >= 4.56.0, <= 5.2.0.
Re-tested in a fresh GPU-enabled env with transformers==5.2.0:
- pytest -q -rs tests/model_executor/test_weight_utils.py -> 2 passed
- pytest -q -rs tests/models/multimodal/processing/test_common.py::test_processing_correctness -k 'AIDC-AI/Ovis2.5-2B or allenai/Molmo2-8B or allenai/Molmo2-O-7B or zai-org/GLM-ASR-Nano-2512' -> 12 passed, 351 deselected

This keeps v4/v5 transition compatibility while explicitly covering 5.2.0 as requested.

Signed-off-by: Codex Bot <codex-bot@users.noreply.github.com>

hmellor · 2026-02-19T10:14:03Z

Thank you for this PR @cccat6, I'm going to ask @zucchini-nlp to help review this so that the processor changes are compatible with the vision for processors in Transformers. Then, when these models are upstreamed into Transformers, we can seamlessly transition to requiring no custom code on the HF Hub or in vLLM

zucchini-nlp · 2026-02-19T10:20:44Z

vllm/transformers_utils/processors/ovis2_5.py

+    missing = [
+        token_name
+        for token_name in REQUIRED_SPECIAL_TOKENS.values()
+        if _get_token_id(tokenizer, token_name) is None
+    ]
+    if not missing:
+        return set()
+
+    add_special_tokens = getattr(tokenizer, "add_special_tokens", None)
+    if callable(add_special_tokens):
+        # Keep order stable so token IDs remain deterministic across runs.
+        add_special_tokens({"additional_special_tokens": missing})
+        missing = [
+            token_name
+            for token_name in missing
+            if _get_token_id(tokenizer, token_name) is None
+        ]
+


this is quite a long of manipulations to get a token-id. TBH I'd expect that if the token id exists, then convert_tokens_to_ids will return an int otherwise we can stop searching

Better to ask for a confirmation from @itazap

zucchini-nlp · 2026-02-19T10:21:55Z

vllm/transformers_utils/processor.py

+def _ensure_processing_utils_compat() -> None:
+    # Some remote processors still import this alias, which is missing in
+    # newer Transformers releases.
+    if not hasattr(hf_processing_utils, "ChatTemplateLoadKwargs"):
+        fallback = getattr(hf_processing_utils, "ProcessorChatTemplateKwargs", None)
+        if fallback is not None:
+            hf_processing_utils.ChatTemplateLoadKwargs = fallback


indeed, I think it was only MiniCPM. Would be great to nudge authors to update their remote code if possible

hmellor

Blocking for now as this PR does a few too many things

hmellor · 2026-02-23T13:07:58Z

requirements/common.txt

 blake3
 py-cpuinfo
-transformers >= 4.56.0, < 5
+transformers >= 4.56.0, <= 5.2.0


We're not ready to do this yet

Suggested change

transformers >= 4.56.0, <= 5.2.0

transformers >= 4.56.0, < 5

hmellor · 2026-02-23T13:08:40Z

vllm/model_executor/model_loader/weight_utils.py

Not related to compatibility, but is a QoL feature. I've extracted this to #35098

hmellor · 2026-02-23T16:23:22Z

vllm/tokenizers/hf.py

Surely turning all the missing methods into noops will break whatever model had them?

hmellor · 2026-02-23T16:44:20Z

Closing as the fixes that are usable have been superseded by other PRs and the ones that remain are not correct

cccat6 requested review from 22quinn, DarkLight1337, njhill and ywang96 as code owners February 18, 2026 02:27

mergify bot added the multi-modality Related to multi-modality (#4194) label Feb 18, 2026

gemini-code-assist bot reviewed Feb 18, 2026

View reviewed changes

Improve Transformers v4/v5 compatibility in tokenizers and processors

54cc897

Signed-off-by: Codex Bot <codex-bot@users.noreply.github.com>

cccat6 force-pushed the pr/transformers-v4-v5-runtime branch from b602bd0 to 54cc897 Compare February 18, 2026 02:42

DarkLight1337 requested a review from hmellor February 18, 2026 03:15

mergify bot added the ci/build label Feb 18, 2026

requirements: allow transformers up to 5.2.0

320ac1a

Signed-off-by: Codex Bot <codex-bot@users.noreply.github.com>

cccat6 force-pushed the pr/transformers-v4-v5-runtime branch from 06ba906 to 320ac1a Compare February 18, 2026 09:23

zucchini-nlp reviewed Feb 19, 2026

View reviewed changes

hmellor mentioned this pull request Feb 19, 2026

Update to transformers v5 #30566

Open

hmellor requested changes Feb 23, 2026

View reviewed changes

hmellor reviewed Feb 23, 2026

View reviewed changes

vllm/tokenizers/hf.py

Copy link

Member

hmellor Feb 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Surely turning all the missing methods into noops will break whatever model had them?

hmellor closed this Feb 23, 2026

Uh oh!

Conversation

cccat6 commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Transformers 5.2.0 validation

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

mergify bot commented Feb 18, 2026

Uh oh!

cccat6 commented Feb 18, 2026

Uh oh!

gaby commented Feb 18, 2026

Uh oh!

cccat6 commented Feb 18, 2026

Uh oh!

cccat6 commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cccat6 commented Feb 18, 2026

Uh oh!

hmellor commented Feb 19, 2026

Uh oh!

zucchini-nlp Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor left a comment

Choose a reason for hiding this comment

Uh oh!

hmellor Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

hmellor commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cccat6 commented Feb 18, 2026 •

edited

Loading

cccat6 commented Feb 18, 2026 •

edited

Loading