fix: AttributeError for Qwen3_omni_moe by Vallabh-1504 · Pull Request #43593 · huggingface/transformers

Vallabh-1504 · 2026-01-29T13:01:01Z

What does this PR do?

This PR fixes a crash when initializing Qwen3OmniMoeTalkerCodePredictorConfig due to a missing attribute reference.

Specifically, it:

Removes the reference to the non-existent use_sliding_window attribute, which was causing an AttributeError.
Adds the missing max_window_layers initialization (defaulting to 28). The existing layer_types logic relies on this attribute, and without it, sliding_window causes a secondary crash.
Adds a new test case (test_code_predictor_config_init) to ensure the configuration initializes correctly.

Fixes #43531

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case. (Issue sliding_window issue with Qwen3-MoE models #43531)
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Rocketknight1 · 2026-01-29T15:06:48Z

cc @vasqu since you commented on the issue!

vasqu

max_window_layers seems like an unnecessary attribute added + let's move the test, please no new files

vasqu · 2026-01-29T15:51:49Z

src/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py

        rope_parameters: int | None = None,
        attention_bias: bool | None = False,
        sliding_window: int | None = None,
+        max_window_layers: int = 28,


Seems irrelevant?

vasqu · 2026-01-29T15:52:09Z

tests/models/qwen3_omni_moe/test_configuration_qwen3_omni_moe.py

The test should be still under test_modeling_xxx

Vallabh-1504 · 2026-01-29T16:34:22Z

hi, @vasqu, thanks for the review.
when I was running pytest I was getting an error regarding max_window_layers.
This attribute is used in the layer_types list comprehension (around line 612).

    def __getattribute__(self, key):
        if key != "attribute_map" and key in super().__getattribute__("attribute_map"):
            key = super().__getattribute__("attribute_map")[key]
>       return super().__getattribute__(key)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       AttributeError: 'Qwen3OmniMoeTalkerCodePredictorConfig' object has no attribute 'max_window_layers'

src\transformers\configuration_utils.py:163: AttributeError

vasqu · 2026-01-29T16:59:28Z

src/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py

        self.intermediate_size = intermediate_size
        self.num_hidden_layers = num_hidden_layers
        self.num_attention_heads = num_attention_heads
-        self.sliding_window = sliding_window if self.use_sliding_window else None


Ok I checked what happened and it seems it was unintentionally added in #41541, but since I'm not super familiar with this model I'd rather wait for @zucchini-nlp to answer here

Imo we should just do self.sliding_window = sliding_window (use_sliding_window was never used at all and should be removed from the docstring) - max_window_layers should be removed alongside it (not reintroduced)

Also the changes should be done in modular and then reapplied via python utils/modular_model_converter.py qwen3_omni_moe

Indeed a bad copy, no need for use_sliding_window. Model always uses sliding layers together with full attention

zucchini-nlp · 2026-01-30T08:46:53Z

src/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py

        **kwargs,
    ):
        self.sliding_window = sliding_window
+        self.max_window_layers = max_window_layers


it is a recurring pattern in many qwen-multimodal models and I think authors incorrectly copied it when adding Qwen3-Omni-MoE. The official checkpoints have max_window_layers saved in config therefore we don't see errore at inference

IMO we need to keep it to match the model's default behavior and to match with docstring

zucchini-nlp · 2026-02-03T09:41:41Z

@Vallabh-1504 hey, any updates on the PR? Would be great to merge it before the next planned release, which I believe should be around this week

Vallabh-1504 · 2026-02-03T14:54:32Z

hi @zucchini-nlp, I was working on it but hitting a bit of a wall.

I was applying changes through modular_qwen3_omni_moe.py, as previously i did manual changes which was wrong. but for some reason, the modular_model_converter.py keeps regenerating the use_sliding_window logic in the final config, even after i removed it from modular_qwen3_omni_moe.py.

kinda stuck in a loop where the generator keeps the code i'm trying to remove.

I also don't know what to do with the max_window_layers, as this is also not initialized anywhere.

Can you guide me through this. If i'm missing a step or if there's some cache with the converter?

zucchini-nlp · 2026-02-03T15:04:52Z

Smth like this should work, we need to delete unused attributed explicitly and re-assign the "differing" attr. For max_window_layers, let's keep it without deleting

https://github.com/Vallabh-1504/transformers/blob/9041720191cdef9348f09f7c1695db606223d353/src/transformers/models/qwen3_omni_moe/modular_qwen3_omni_moe.py#L571-L572

Vallabh-1504 · 2026-02-03T16:55:39Z

@zucchini-nlp, I have applied the changes as requested!

zucchini-nlp

One tiny comment and let's wait for @vasqu's review

zucchini-nlp · 2026-02-03T17:31:08Z

src/transformers/models/qwen3_omni_moe/configuration_qwen3_omni_moe.py

        self.num_code_groups = num_code_groups
        self.vocab_size = vocab_size
        self.max_position_embeddings = max_position_embeddings
        self.hidden_size = hidden_size
        self.intermediate_size = intermediate_size
        self.num_hidden_layers = num_hidden_layers
        self.num_attention_heads = num_attention_heads
-        self.sliding_window = sliding_window if self.use_sliding_window else None
+        self.sliding_window = sliding_window
+        self.max_window_layers = max_window_layers


nit: duplicate

zucchini-nlp · 2026-02-03T17:31:36Z

src/transformers/models/qwen3_omni_moe/modular_qwen3_omni_moe.py

@@ -473,6 +475,7 @@ def __init__(
        **kwargs,
    ):
        self.sliding_window = sliding_window
+        self.max_window_layers = max_window_layers


not needed, super assigns it already which is why we got duplicates in auto-generated code :)

zucchini-nlp · 2026-02-03T17:35:58Z

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py

+class TestQwen3OmniMoeCodePredictorConfig(unittest.TestCase):
+    def test_code_predictor_config_init(self):
+        """
+        Test that Qwen3OmniMoeTalkerCodePredictorConfig initializes correctly
+        and accepts max_window_layers while removing use_sliding_window.
+        """
+
+        config = Qwen3OmniMoeTalkerCodePredictorConfig(


ideally we need a complete model test for the 'TalkerModel' model. But I realize that the model doesn't follow transformers standards and we'll skip anyway many tests from ModelTesterMixin, or override a lot of them

I'm fine with deleting the current test in that case, @vasqu wdyt?

Imo, we can add a small regression test under

transformers/tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py

Line 61 in d75ecdb

class Qwen3OmniMoeThinkerForConditionalGenerationTester:

at least, we don't need a separate class for this

Just a tad weird because the naming is weird but if we don't test it we ought to repeat it in a refactor 😬

HuggingFaceDocBuilderDev · 2026-02-03T17:40:45Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

vasqu

Don't have much to add onto @zucchini-nlp's comments, I'd just prefer to move the test under the general tester class

vasqu · 2026-02-03T18:29:48Z

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py

+class TestQwen3OmniMoeCodePredictorConfig(unittest.TestCase):
+    def test_code_predictor_config_init(self):
+        """
+        Test that Qwen3OmniMoeTalkerCodePredictorConfig initializes correctly
+        and accepts max_window_layers while removing use_sliding_window.
+        """
+
+        config = Qwen3OmniMoeTalkerCodePredictorConfig(


Imo, we can add a small regression test under

transformers/tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py

Line 61 in d75ecdb

class Qwen3OmniMoeThinkerForConditionalGenerationTester:

at least, we don't need a separate class for this

Just a tad weird because the naming is weird but if we don't test it we ought to repeat it in a refactor 😬

github-actions · 2026-02-04T10:26:19Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

vasqu reviewed Jan 29, 2026

View reviewed changes

zucchini-nlp reviewed Jan 30, 2026

View reviewed changes

Sebmono mentioned this pull request Feb 1, 2026

[PR] fix: AttributeError for Qwen3_omni_moe Sandgarden-Demo/transformers#50

Open

5 tasks

zucchini-nlp mentioned this pull request Feb 3, 2026

Add Qwen3-Omni model registration to AutoModel and AutoModelForConditionalGeneration classes #43684

Closed

zucchini-nlp added the for patch Tag issues / labels that should be included in the next patch label Feb 3, 2026

correct-fix-through-modular

d75ecdb

Vallabh-1504 force-pushed the fix-attribute-error branch from 9041720 to d75ecdb Compare February 3, 2026 16:51

zucchini-nlp approved these changes Feb 3, 2026

View reviewed changes

vasqu approved these changes Feb 3, 2026

View reviewed changes

lvliang-intel mentioned this pull request Feb 4, 2026

[Feature]: Support Omni model quantization intel/auto-round#1387

Open

fix-duplicate

2d186c4

zucchini-nlp enabled auto-merge (squash) February 4, 2026 10:36

zucchini-nlp merged commit 257a602 into huggingface:main Feb 4, 2026
19 checks passed

Conversation

Vallabh-1504 commented Jan 29, 2026

What does this PR do?

Before submitting

Who can review?

Uh oh!

Rocketknight1 commented Jan 29, 2026

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Vallabh-1504 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zucchini-nlp commented Feb 3, 2026

Uh oh!

Vallabh-1504 commented Feb 3, 2026

Uh oh!

zucchini-nlp commented Feb 3, 2026

Uh oh!

Vallabh-1504 commented Feb 3, 2026

Uh oh!

zucchini-nlp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented Feb 3, 2026

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Vallabh-1504 commented Jan 29, 2026 •

edited

Loading

zucchini-nlp Jan 30, 2026 •

edited

Loading