Fix few issues in Qwen_3_Omni_Moe by Sai-Suraj-27 · Pull Request #44848 · huggingface/transformers

Sai-Suraj-27 · 2026-03-19T07:30:39Z

What does this PR do?

Update Qwen3_Omni_Moe, to fix these attribute errors Qwen3OmniModelIntegrationTests

Almost same issue was fixed initally in #43084 but the config refactor in #41250 dropped/missed the initializer_range from Qwen3OmniMoeConfig.

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline,
Pull Request section?
Was this discussed/approved via a Github issue or the forum? Please add a link
to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

@Rocketknight1 @vasqu

vasqu · 2026-03-19T07:59:42Z

run-slow: qwen3_omni_moe

vasqu

Thanks, checking with CI 🫡

github-actions · 2026-03-19T08:01:07Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

vasqu · 2026-03-19T08:38:55Z

Ok it fixes one issue and reveals some other ones 😓 can you recheck or rather rename the PR since it does unblock partially at least

Slow tests still fail

github-actions · 2026-03-19T09:28:38Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	49a84fd9	workflow commit (merge commit)
PR	f4a7c27a	branch commit (from PR)
main	529504b2	base commit (on `main`)

Model CI Report

❌ 3 new failed tests from this PR 😭

qwen3_omni_moe:
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_w_audio (❌ ⟹ ❌)

…wen3_omni_config

Sai-Suraj-27 · 2026-03-19T14:16:55Z

Ok it fixes one issue and reveals some other ones 😓 can you recheck or rather rename the PR since it does unblock partially at least

Hey, @vasqu. Gave fix to this one, atleast this multiple values for argument 'next_sequence_length' error should be gone now. Can you check & trigger run-slow 👀.

vasqu · 2026-03-19T14:53:22Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-19T14:54:41Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

github-actions · 2026-03-19T16:30:22Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	d6da4c2c	workflow commit (merge commit)
PR	68ca6b4a	branch commit (from PR)
main	be8d8a4c	base commit (on `main`)

Model CI Report

❌ 3 new failed tests from this PR 😭

qwen3_omni_moe:
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_w_audio (❌ ⟹ ❌)

…wen3_omni_config

Sai-Suraj-27 · 2026-03-22T10:40:30Z

Hey, Pushed a potential fix for these. I think The _no_split_modules should cover the full AudioEncoder & VisionEncoder so that device_map="auto" will not keep child modules of Qwen3OmniMoeAudioEncoder on separate devices incase of multi-gpu env. Followed similar to Qwen2_5Omni.

vasqu · 2026-03-25T13:31:35Z

run-slow: qwen3_omni_moe

vasqu · 2026-03-25T13:32:02Z

Sorry, I was off for a few days. Now back 🤗 @Sai-Suraj-27 checking run-slow

github-actions · 2026-03-25T13:32:49Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

github-actions · 2026-03-25T14:08:37Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	3bf2b531	workflow commit (merge commit)
PR	a04a9b98	branch commit (from PR)
main	28af8184	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

vasqu · 2026-03-25T14:18:53Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-25T14:20:39Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

HuggingFaceDocBuilderDev · 2026-03-25T14:30:54Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

github-actions · 2026-03-25T14:57:44Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	c5f85c8b	workflow commit (merge commit)
PR	28d3bd61	branch commit (from PR)
main	2f624917	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

vasqu · 2026-03-25T20:57:51Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-25T20:59:08Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

github-actions · 2026-03-25T21:43:34Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	3d85820e	workflow commit (merge commit)
PR	47123f8d	branch commit (from PR)
main	c9faacd7	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

ydshieh · 2026-03-26T14:29:25Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-26T14:30:40Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

github-actions · 2026-03-26T14:44:44Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	11fa0cf6	workflow commit (merge commit)
PR	22f647b1	branch commit (from PR)
main	da37a4d9	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

ydshieh · 2026-03-27T12:41:45Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-27T12:42:53Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

ydshieh · 2026-03-27T13:25:15Z

no longer crash, but just

FAILED tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test - AssertionError: "user[101 chars]on, here is a breakdown of what you're hearing and seeing:\n\n" != "user[101 chars]on, here is a breakdown of what you're hearing and seeing:-"
  user
  What's that sound and what kind of dog is this?
  assistant
- Based on the audio and visual information, here is a breakdown of what you're hearing and seeing:
?                                                                                                  ^
+ Based on the audio and visual information, here is a breakdown of what you're hearing and seeing:-?                                                                                                  ^
-
FAILED tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch - AssertionError: Lists differ: ["use[99 chars]ation provided:\n\nThe sound you hear is the d[191 chars]hed"] != ["use[99 chars]ation, here is a breakdown of what you're hear[187 chars]n\n"]

First differing element 0:
"user[98 chars]ation provided:\n\nThe sound you hear is the d[17 chars]ched"
"user[98 chars]ation, here is a breakdown of what you're hear[15 chars]\n\n"

Diff is 672 characters long. Set self.maxDiff to None to see it.
FAILED tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_w_audio - RuntimeError: Tensor on device meta is not on the expected device cuda:0!

vasqu · 2026-03-27T13:31:27Z

Yea, seems reasonable - the test didn't run at all before and crashed, this PR at least let's the integration tests produce output again @ydshieh

Can we change the title tho @Sai-Suraj-27? Also looks like the meta device one is not specific to the multi-gpu case (which we talked about before)

Sai-Suraj-27 · 2026-03-27T14:11:53Z

Yea, seems reasonable - the test didn't run at all before and crashed, this PR at least let's the integration tests produce output again @ydshieh

Can we change the title tho @Sai-Suraj-27? Also looks like the meta device one is not specific to the multi-gpu case (which we talked about before)

Yes, but for the text expectation mismatch failures, should I try & update the expected text maybe?

vasqu · 2026-03-27T14:15:04Z

Nope, not for now - imo I would like to have a failure for now / xmark. Something somewhere changed and arbitrarily changing the values is not good

vasqu · 2026-03-27T14:19:51Z

Do you want to investigate the meta device issue? Otherwise, I would merge as is for now

Sai-Suraj-27 · 2026-03-27T14:22:20Z

Do you want to investigate the meta device issue? Otherwise, I would merge as is for now

Sure, Let me check that over the weekend.

github-actions · 2026-03-27T15:52:02Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	b57b7dab	workflow commit (merge commit)
PR	22f647b1	branch commit (from PR)
main	7b00e3ba	base commit (on `main`)

Model CI Report

❌ 3 new failed tests from this PR 😭

qwen3_omni_moe:
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_batch (❌ ⟹ ❌)
tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py::Qwen3OmniModelIntegrationTest::test_small_model_integration_test_w_audio (❌ ⟹ ❌)

ydshieh · 2026-03-30T14:43:55Z

run-slow: qwen3_omni_moe

github-actions · 2026-03-30T14:44:39Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

ydshieh · 2026-03-30T14:45:03Z

@Sai-Suraj-27 To move fast, I pushed some commits that should work well (on our CI runner), including the fixes for meta device.

I will merge once @vasqu have a final look 🙏 .

Thanks again for the work !

github-actions · 2026-03-30T14:45:11Z

Workflow Run ⚙️

This comment contains run-slow, running the specified jobs:

models: ["models/qwen3_omni_moe"]
quantizations: []

vasqu

Thanks, will also try to investigate a bit more later because something clearly goes wrong within the model

vasqu · 2026-03-30T14:48:36Z

tests/models/qwen3_omni_moe/test_modeling_qwen3_omni_moe.py

        )
        self.assertFalse(torch.isnan(output[1]).any().item())

+    @run_first


Any reason we want this to run first?

vasqu · 2026-03-30T14:50:19Z

src/transformers/generation/utils.py


        if "inputs_embeds" in model_kwargs:
-            return torch.ones((batch_size, 0), dtype=torch.long, device=self.device)
+            return torch.ones(


Can you add comment with reference to here

ohoh yes, forgot before push

Hey, @ydshieh. Thanks for pushing the fix. I was able to run the test on RTX PRO 6000, & it's running fine without the meta device issue. But incase of A-10 GPU the device_map="auto" is offloading the talker module to CPU & iiuc from accelerate code, it keeps the parameters of cpu/disk offloaded modules as meta tensors (which is why model.talker.device is giving "meta" in case of A10) & only loads the real-weights on to the GPU later just before forward.

Since the test ran fine on the big gpu but failing on A10, I think, I can confrim with this fix & that the issue is with how we are using self.device in this method. So, Maybe we can add a comment regarding this accelerate behaviour here pointing to this accelerate code.

Hi @Sai-Suraj-27

a comment is added (a few line below)

# Use the device of the existing tensor to avoid any potential `meta` device isssue. # See PR #44848. (Previously, it used `self.device`.)

I think it's enough with the reference to this PR.

github-actions · 2026-03-30T14:53:03Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

github-actions · 2026-03-30T15:06:01Z

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=44848&sha=0865d5

github-actions · 2026-03-30T15:08:04Z

CI Results

Workflow Run ⚙️

Commit Info

Context	Commit	Description
RUN	e36b0474	workflow commit (merge commit)
PR	09d23fa4	branch commit (from PR)
main	02063e68	base commit (on `main`)

✅ No failing test specific to this PR 🎉 👏 !

github-actions · 2026-03-30T15:09:26Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

github-actions · 2026-03-30T15:26:57Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: qwen3_omni_moe

github-actions · 2026-03-30T15:45:27Z

View the CircleCI Test Summary for this PR:

https://huggingface.co/spaces/transformers-community/circle-ci-viz?pr=44848&sha=f57a22

Fix Qwen3OmniMoeConfig has no attribute initializer_range

f4a7c27

vasqu previously approved these changes Mar 19, 2026

View reviewed changes

Sai-Suraj-27 added 2 commits March 19, 2026 14:11

Merge branch 'main' of github.com:huggingface/transformers into fix_q…

bf06736

…wen3_omni_config

Fix passing of args

68ca6b4

Merge branch 'main' of github.com:huggingface/transformers into fix_q…

d7a6fb3

…wen3_omni_config

Fix no_split_modules

a04a9b9

Merge branch 'main' into fix_qwen3_omni_config

28d3bd6

vasqu mentioned this pull request Mar 25, 2026

Add cuda compatibility check for using grouped_mm #45001

Open

6 tasks

Merge branch 'main' into fix_qwen3_omni_config

47123f8

Sai-Suraj-27 changed the title ~~Fix failing Qwen3OmniModelIntegrationTests~~ Fix few issues in Qwen_3_Omni_Moe Mar 27, 2026

ydshieh added 2 commits March 30, 2026 16:31

fix and improve

45fbffe

format

09d23fa

vasqu approved these changes Mar 30, 2026

View reviewed changes

fix modular

0865d56

fix modular

3b4581a

comment

f57a226

ydshieh merged commit 813c7c6 into huggingface:main Mar 30, 2026
27 of 29 checks passed

Conversation

Sai-Suraj-27 commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

vasqu commented Mar 19, 2026

Uh oh!

vasqu left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Mar 19, 2026

Uh oh!

vasqu commented Mar 19, 2026

Uh oh!

github-actions bot commented Mar 19, 2026

CI Results

Commit Info

Model CI Report

Uh oh!

Sai-Suraj-27 commented Mar 19, 2026

Uh oh!

vasqu commented Mar 19, 2026

Uh oh!

github-actions bot commented Mar 19, 2026

Uh oh!

github-actions bot commented Mar 19, 2026

CI Results

Commit Info

Model CI Report

Uh oh!

Sai-Suraj-27 commented Mar 22, 2026

Uh oh!

vasqu commented Mar 25, 2026

Uh oh!

vasqu commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

CI Results

Commit Info

Uh oh!

vasqu commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

HuggingFaceDocBuilderDev commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

CI Results

Commit Info

Uh oh!

vasqu commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

Uh oh!

github-actions bot commented Mar 25, 2026

CI Results

Commit Info

Uh oh!

ydshieh commented Mar 26, 2026

Uh oh!

github-actions bot commented Mar 26, 2026

Uh oh!

github-actions bot commented Mar 26, 2026

CI Results

Commit Info

Uh oh!

ydshieh commented Mar 27, 2026

Uh oh!

github-actions bot commented Mar 27, 2026

Uh oh!

ydshieh commented Mar 27, 2026

Uh oh!

vasqu commented Mar 27, 2026

Uh oh!

Sai-Suraj-27 commented Mar 27, 2026

Uh oh!

Sai-Suraj-27 commented Mar 19, 2026 •

edited

Loading

Sai-Suraj-27 Mar 30, 2026 •

edited

Loading