Fix vllm cis by ArthurZucker · Pull Request #45139 · huggingface/transformers

ArthurZucker · 2026-03-31T09:26:28Z

What does this PR do?

More fixes

ArthurZucker · 2026-03-31T09:40:02Z

        This is very important as most embeddings are tied, and they are huge params (vocabularies are often 256k), so
        running inits on them is very costly."""
-        for tied_param in self.all_tied_weights_keys.keys():
+        for tied_param in getattr(self, "all_tied_weights_keys", {}).keys():


fix remote code

ArthurZucker · 2026-03-31T09:40:08Z

-            token_value = self._special_tokens_map.get(key_without_id)
+            # Use __dict__.get to avoid recursive __getattr__ when _special_tokens_map
+            # is not yet initialized (e.g. during fast tokenizer __init__)
+            token_value = self.__dict__.get("_special_tokens_map", {}).get(key_without_id)


same easy fix for remote code

ArthurZucker · 2026-03-31T09:40:29Z

 # from the model config. You can append new {'rope_type': callable} pairs to this rope_parameters to enable custom RoPE
 # parameterizations, as long as the callable has the same signature.
 ROPE_INIT_FUNCTIONS = {
+    "default": _compute_default_rope_parameters,


remote code BC?

cc @zucchini-nlp

It is true that remote code won't have it, but we likely would also need to refactor a lot of models, seems risky; especially for models that do have a different default init so we need to check if some code exists first and then use it as fallback

HuggingFaceDocBuilderDev · 2026-03-31T10:13:40Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

…llm-cis

ArthurZucker · 2026-04-02T15:16:36Z

run-slow: aimv2, altclip, audioflamingo3, bart, beit, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, blenderbot, blenderbot_small, blip, bridgetower, camembert, clip

github-actions · 2026-04-02T15:17:53Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/aimv2", "models/altclip", "models/audioflamingo3", "models/bart", "models/beit", "models/bert", "models/bert_generation", "models/big_bird", "models/bigbird_pegasus", "models/biogpt", "models/blenderbot", "models/blenderbot_small", "models/blip", "models/bridgetower", "models/camembert", "models/clip"]
quantizations: []

vasqu

Lgtm, most is nit except for 2

Whisper/audio flamingo seems weird to me re consuming bsz / leaving bsz optionally
RoPE default init --> does this still call for example ernie vl default compute; would likely need to check def _init_weights here what happens

vasqu · 2026-04-02T15:20:48Z

 # from the model config. You can append new {'rope_type': callable} pairs to this rope_parameters to enable custom RoPE
 # parameterizations, as long as the callable has the same signature.
 ROPE_INIT_FUNCTIONS = {
+    "default": _compute_default_rope_parameters,


cc @zucchini-nlp

It is true that remote code won't have it, but we likely would also need to refactor a lot of models, seems risky; especially for models that do have a different default init so we need to check if some code exists first and then use it as fallback

vasqu · 2026-04-02T15:21:42Z

    def _validate_default_rope_parameters(self, rope_parameters: dict, ignore_keys: set | None = None):
-        required_keys = {"rope_type", "rope_theta"}
+        required_keys = {"rope_type"}
+        optional_keys = {"rope_theta"}


likely needs to be done elsewhere then too? I think all need rope theta?

vasqu · 2026-04-02T15:22:26Z

+class ConfigSubclassKwOnlyTest(unittest.TestCase):
+    """Test that config subclasses with non-default fields following parent default fields
+    no longer raise TypeError (fixed by kw_only=True in __init_subclass__). Regression
+    test for https://github.com/huggingface/transformers/issues/XXXX."""


Any issue there we could link or an example?

github-actions · 2026-04-02T15:44:00Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	3c8d59b1	workflow commit (merge commit)
PR	08893759	branch commit (from PR)
main	a594e09e	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

ArthurZucker · 2026-04-07T13:44:18Z

+        config = LlamaConfig()
+        config.rope_parameters = {"rope_type": "default", "rope_theta": 10000.0, "unknown_param": 1}
+        with self.assertLogs("transformers.modeling_rope_utils", level="WARNING"):
+            config.validate_rope()


cc @zucchini-nlp IDK if that's okay with you?

yep, tbh i thought it was already a warning, instead of error in the past. Maybe smth changed recently

ArthurZucker · 2026-04-07T15:22:28Z

run-slow: aimv2, altclip, audioflamingo3, bart, beit, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, blenderbot, blenderbot_small, blip, bridgetower, camembert, clip

github-actions · 2026-04-07T15:23:42Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/aimv2", "models/altclip", "models/audioflamingo3", "models/bart", "models/beit", "models/bert", "models/bert_generation", "models/big_bird", "models/bigbird_pegasus", "models/biogpt", "models/blenderbot", "models/blenderbot_small", "models/blip", "models/bridgetower", "models/camembert", "models/clip"]
quantizations: []

zucchini-nlp · 2026-04-07T15:34:18Z

+        required_keys = {"rope_type"}
+        optional_keys = {"rope_theta"}


yep, this kinda defeats the point of validation because a RoPE dict with no theta isn't valid for our modules

yeah but we always default to default_theta if its not there no?

validation always happens after teh defaults are set, so ideally it shouldn't raise an error. Do we know why the theta was missing?

zucchini-nlp · 2026-04-07T15:36:42Z

+            if rope_type == "default":
+                continue  # "default" is always valid with just rope_theta


default isn't in all_rope_types anyway, no?

github-actions · 2026-04-07T15:45:39Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	5a8a8309	workflow commit (merge commit)
PR	f924677b	branch commit (from PR)
main	1897dd05	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

- Add _compute_default_rope_parameters and register as ROPE_INIT_FUNCTIONS["default"] - Accept ignore_keys kwarg in validate_rope for backward compat with vllm - Remove RopeDefaultTypeTest class (redundant with existing tests) - Fix test_rope_validation to skip "default" rope type in KeyError checks Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

…ion docstrings rope_theta defaults to default_theta when omitted from serialized configs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

github-actions · 2026-04-08T08:32:05Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: aimv2, altclip, audioflamingo3, bart, beit, bert, bert_generation, big_bird, bigbird_pegasus, biogpt, blenderbot, blenderbot_small, blip, bridgetower, camembert, clip

ArthurZucker

Checked:

class RotaryEmbeddingConfigMixin:
    """
    A Mixin containing the functionality to standardize and validate RoPE parameters.
    """

    default_theta = 10_000.0

faire to say its optional

* fixes * safe linspace? * up * up * remove batch * nits * style * fix repo * more * update * update * revert * up * fix copy * work? * adress some of the comments * update * fixes * fix-repo! * styling post merge * don't pass default * up * up * push * fix vllm compat: add default rope type and ignore_keys support - Add _compute_default_rope_parameters and register as ROPE_INIT_FUNCTIONS["default"] - Accept ignore_keys kwarg in validate_rope for backward compat with vllm - Remove RopeDefaultTypeTest class (redundant with existing tests) - Fix test_rope_validation to skip "default" rope type in KeyError checks Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * revert rope utils additions, keep only test removal Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> * docs: mark rope_theta as optional in RopeParameters and compute function docstrings rope_theta defaults to default_theta when omitted from serialized configs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

fixes

f36ea56

ArthurZucker force-pushed the fix-vllm-cis branch from 8318200 to f36ea56 Compare March 31, 2026 09:36

ArthurZucker commented Mar 31, 2026

View reviewed changes

ArthurZucker added 3 commits March 31, 2026 11:50

safe linspace?

013f76a

up

7a6f698

up

04fb689

ArthurZucker added 5 commits March 31, 2026 13:24

remove batch

1b53a3f

nits

4895e1f

style

b3febee

fix repo

411eb60

more

55809c8

hmellor linked an issue Apr 1, 2026 that may be closed by this pull request

[Transformers v5] IsaacForConditionalGeneration vllm-project/vllm#38389

Open

hmellor mentioned this pull request Apr 1, 2026

Update to transformers v5 vllm-project/vllm#30566

Merged

ArthurZucker added 7 commits April 2, 2026 08:15

Merge branch 'main' of github.com:huggingface/transformers into fix-v…

01cdc60

…llm-cis

update

0d8628b

update

d3484b7

revert

60fd4a1

up

98ae737

fix copy

814c3a8

work?

0889375

ArthurZucker marked this pull request as ready for review April 2, 2026 15:15

vasqu approved these changes Apr 2, 2026

View reviewed changes

ArthurZucker added 4 commits April 7, 2026 11:54

adress some of the comments

8ea59cb

update

1852250

fixes

9a77627

fix-repo!

db0529e

ArthurZucker and others added 2 commits April 7, 2026 12:11

Merge branch 'main' into fix-vllm-cis

1d81931

styling post merge

a104d58

ArthurZucker commented Apr 7, 2026

View reviewed changes

ArthurZucker added 4 commits April 7, 2026 17:04

don't pass default

30f6d73

up

12bf667

up

907402a

push

f924677

zucchini-nlp reviewed Apr 7, 2026

View reviewed changes

ArthurZucker and others added 3 commits April 8, 2026 07:56

revert rope utils additions, keep only test removal

ae4860e

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

docs: mark rope_theta as optional in RopeParameters and compute funct…

c8985c5

…ion docstrings rope_theta defaults to default_theta when omitted from serialized configs. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

ArthurZucker commented Apr 8, 2026

View reviewed changes

ArthurZucker merged commit d081c71 into main Apr 8, 2026
28 of 30 checks passed

ArthurZucker deleted the fix-vllm-cis branch April 8, 2026 11:19

		if rope_type == "default":
		continue # "default" is always valid with just rope_theta

Conversation

ArthurZucker commented Mar 31, 2026

What does this PR do?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Mar 31, 2026

Uh oh!

ArthurZucker commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 2, 2026

CI Results

Commit Info

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ArthurZucker commented Apr 7, 2026

Uh oh!

github-actions bot commented Apr 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 7, 2026

CI Results

Commit Info

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

ArthurZucker left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants