[Bug fix] fix Qwen3VLMoe

[Wangzheee]

SUMMARY:
"please provide a brief summary"
Fix insufficient CPU memory: Do not create a new object to prevent insufficient CPU memory

[gemini-code-assist]

Summary of Changes

Hello @Wangzheee, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a bug within the Qwen3VLMoe model's calibration process. The changes primarily focus on optimizing memory usage during the initialization of expert modules and ensuring that the moe_intermediate_size configuration is correctly applied. Additionally, it includes a minor API adjustment for clarity and improves the robustness of weight assignment within the model.

Highlights

• Memory Optimization: Refactored the initialization of CalibrateQwen3VLMoeTextSparseMoeBlock to prevent excessive CPU memory usage by avoiding the creation of a new object list, instead iteratively appending expert modules.
• Configuration Alignment: Updated the Qwen3VLMoeTextMLP constructor call to explicitly pass config.moe_intermediate_size, ensuring correct configuration for the Mixture-of-Experts (MoE) layers.
• API Refinement: Corrected the API signature and parameter passing in the replace function for CalibrateQwen3VLMoeTextSparseMoeBlock by renaming the original parameter to module for clarity.
• Parameter Handling: Changed how weights are assigned for gate_proj, up_proj, and down_proj to use torch.nn.Parameter directly with requires_grad=False, which is a more robust method for managing model parameters.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request addresses a bug in Qwen3VLMoe calibration. The changes primarily refactor the initialization of SequentialQwen3VLMoeTextExperts to be more memory-efficient by avoiding the creation of intermediate tensor copies, which aligns with the goal of preventing CPU memory issues. The API for Qwen3VLMoeTextMLP is also correctly updated. The changes are logical and well-motivated. I have one suggestion to improve code style and readability for better long-term maintenance.

[gemini-code-assist]

For better readability and adherence to Python style conventions (PEP 8), it's recommended to use more descriptive variable names and standard spacing.

• num_exp and inter are a bit cryptic. Using num_experts and intermediate_size (as was used in the previous version of the code) makes the code easier to understand.
• The extra spacing used for aligning = operators is not standard and can make code harder to maintain. A single space around operators is preferred.

The suggested change applies these improvements.

Suggested change

	num_exp = original.gate_up_proj.shape[0]
	inter = original.down_proj.shape[1]
	for i in range(num_exp):
	gate_up = original.gate_up_proj[i]
	down = original.down_proj[i]
	mlp = Qwen3VLMoeTextMLP(config, config.moe_intermediate_size)
	mlp.gate_proj.weight = torch.nn.Parameter(
	gate_up[:, :inter].t(), requires_grad=False)
	mlp.up_proj.weight = torch.nn.Parameter(
	gate_up[:, inter:].t(), requires_grad=False)
	mlp.down_proj.weight = torch.nn.Parameter(
	down.t(), requires_grad=False)
	num_experts = original.gate_up_proj.shape[0]
	intermediate_size = original.down_proj.shape[1]
	for i in range(num_experts):
	gate_up = original.gate_up_proj[i]
	down = original.down_proj[i]
	mlp = Qwen3VLMoeTextMLP(config, config.moe_intermediate_size)
	mlp.gate_proj.weight = torch.nn.Parameter(
	gate_up[:, :intermediate_size].t(), requires_grad=False)
	mlp.up_proj.weight = torch.nn.Parameter(
	gate_up[:, intermediate_size:].t(), requires_grad=False)
	mlp.down_proj.weight = torch.nn.Parameter(
	down.t(), requires_grad=False)

[github-actions]

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

Note: This is required to complete the testing suite, please only add the label once the PR is code complete and local testing has been performed.

[dsikka]

Hi @Wangzheee we recently updated this definition, do you mind rebasing?

[Wangzheee]

Hi @Wangzheee we recently updated this definition, do you mind rebasing?

Thanks~
I rebased the latest code for main. It still seems like I need to use this PR to fix insufficient CPU memory.

[brian-dellabetta]

These changes make sense to me, though deferring to the VL expert -- @kylesayrs WDYT?

[dsikka]

We should test these with existing examples before merge

[Wangzheee]

We should test these with existing examples before merge

Thanks～
When quantifying Qwen3 vl, we found that a particularly large amount of CPU memory is allocated here