Skip to content

[Bugfix] fix wrong partial_rotary_factor calculation for bailing_moe model. #43770

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ywang96 merged 3 commits into from
Jun 1, 2026
+6 −1
Merged

[Bugfix] fix wrong partial_rotary_factor calculation for bailing_moe model. #43770

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c06ffd5
fix(bailing_moe): fix rotary_dim calculation logic
zzt93 May 27, 2026
cecd120
Merge branch 'main' into fix-bailing
zzt93 May 28, 2026
6e1fa91
Merge branch 'main' into fix-bailing
ZJY0516 Jun 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion vllm/model_executor/models/bailing_moe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,12 @@ def __init__(
prefix=f"{prefix}.dense",
)

rotary_dim = getattr(config, "rotary_dim", self.head_dim)
rotary_dim = getattr(config, "rotary_dim", None)
if rotary_dim is None:
partial_rotary_factor = getattr(config, "partial_rotary_factor", 1.0)
rotary_dim = int(self.head_dim * partial_rotary_factor)
if rotary_dim is None:
Copy link
Copy Markdown

@xianbaoqian xianbaoqian May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check seems to be a dead code?

Copy link
Copy Markdown
Contributor Author

@zzt93 zzt93 May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, for the current config.json. But I'm not sure whether it will change, and cause crash for line 139, so add this defense code.

Copy link
Copy Markdown
Contributor Author

@zzt93 zzt93 May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fine to delete it if needed.

rotary_dim = self.head_dim
config.rope_parameters["partial_rotary_factor"] = rotary_dim / self.head_dim

self.rotary_emb = get_rope(
Expand Down
Loading