[Bugifx] qwen3 rope parameter compatibility

[lviy]

Motivation

To fix #20932
SGLang fails to load some Qwen3 MoE checkpoints whose HF config uses top-level rope_theta (v4-style) but does not define
rope_parameters (v5-style).

Modifications

This PR updated qwen3moe.py to make RoPE config parsing compatible across checkpoint formats: it now checks whether RoPE is provided as config.rope_parameters (new style) or as top-level config.rope_theta/config.rope_scaling (old style), and uses the available field accordingly. This prevents load failures caused by different RoPE parameter naming in Qwen3 configs.

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.

Review Process

1. Ping Merge Oncalls to start the PR flow. See the PR Merge Process.
2. Get approvals from CODEOWNERS and other reviewers.
3. Trigger CI tests with comments or contact authorized users to do so.
   • /tag-run-ci-label, /rerun-failed-ci, /tag-and-rerun-ci
4. After green CI and required approvals, ask Merge Oncalls to merge.

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the compatibility of SGLang with Qwen3 MoE models by addressing inconsistencies in how RoPE (Rotary Positional Embeddings) parameters are defined in model configurations. The changes ensure that the system can correctly parse and apply RoPE settings from various Qwen3 checkpoint formats, thereby preventing loading errors and improving the overall robustness of model integration.

Highlights

• RoPE Parameter Compatibility: Resolved an issue where SGLang failed to load certain Qwen3 MoE checkpoints due to inconsistent RoPE parameter naming conventions in their Hugging Face configurations.
• Flexible RoPE Config Parsing: Implemented logic in qwen3_moe.py to dynamically check for RoPE parameters, supporting both the newer config.rope_parameters and older top-level config.rope_theta/config.rope_scaling formats.
• Enhanced Model Loading Robustness: Prevented load failures by ensuring the system can correctly identify and utilize RoPE configuration, regardless of how it's structured within the Qwen3 model's configuration.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a fix for RoPE parameter compatibility in Qwen3 models, allowing them to load from different checkpoint formats. The logic correctly handles both newer rope_parameters and older rope_theta/rope_scaling configurations. My review identified a critical issue where the calculated rope_theta is not stored as an instance variable, which would lead to a runtime error in apply_qk_norm_rope. I have provided a code suggestion to fix this.

[lviy]

store rope_theta

[JustinTong0323]

Use

sglang/python/sglang/srt/utils/hf_transformers_utils.py

Lines 133 to 144 in 4c52b7f

	def get_rope_config(config):
	"""Get (rope_theta, rope_scaling) from config, supporting both v4 and v5.
	In transformers v5, rope_theta/rope_scaling are accessed via the computed
	property config.rope_parameters. Trust-remote-code configs or parent configs
	passed to sub-models may not have this property or may return None.
	Falls back to the v4-style config.rope_theta / config.rope_scaling attributes.
	"""
	rope_params = getattr(config, "rope_parameters", None)
	if rope_params is not None:
	return rope_params["rope_theta"], rope_params
	return config.rope_theta, getattr(config, "rope_scaling", None)

[JustinTong0323]

/tag-run-ci-label

[lviy]

Thanks for your update, it looks good!

[JustinTong0323]

/tag-and-rerun-ci

[lviy]

cc @FortPercent @qzweng