Skip to content

Add mm_token_type_ids to qwen3_5_moe lce_forward#1140

Merged
Tcc0403 merged 1 commit intolinkedin:mainfrom
albertvillanova:fu-1120
Mar 12, 2026
Merged

Add mm_token_type_ids to qwen3_5_moe lce_forward#1140
Tcc0403 merged 1 commit intolinkedin:mainfrom
albertvillanova:fu-1120

Conversation

@albertvillanova
Copy link
Contributor

@albertvillanova albertvillanova commented Mar 10, 2026

Summary

This PR fixes the lce_forward function for qwen3_5_moe model, adding support for mm_token_type_ids optional parameter related to multimodal processing.

Follow-up to:

This fixes a ValueError in model.generate() with transformers > 5.2.0, after they merged:

See related issue downstream in TRL:

Testing Done

  • Hardware Type:
  • run make test to ensure correctness
  • run make checkstyle to ensure code style
  • run make test-convergence to ensure convergence

Copy link
Collaborator

@Tcc0403 Tcc0403 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you

@Tcc0403 Tcc0403 added this pull request to the merge queue Mar 12, 2026
Merged via the queue into linkedin:main with commit e6f84b8 Mar 12, 2026
5 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants