Skip to content

[Bugfix] Fix some issues with MoERunner PR #32344 #34371

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vllm-bot merged 4 commits into from
Feb 11, 2026
Merged

[Bugfix] Fix some issues with MoERunner PR #32344 #34371

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
c1644f8
[Bugfix] Move ensure_moe_quant_config_init calls inside moe_forward ops
bnellnm Feb 11, 2026
1ddd73e
fix gate handling
bnellnm Feb 11, 2026
26605df
add comments
bnellnm Feb 11, 2026
2e79ded
Merge branch 'main' into move-init-call
bnellnm Feb 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions vllm/model_executor/layers/fused_moe/layer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,7 +700,7 @@ def layer_id(self):

@property
def gate(self) -> torch.nn.Module | None:
return self._gate
return self._gate if self.use_overlapped else None

@property
def tp_size(self):
Expand All @@ -725,7 +725,7 @@ def use_ep(self):
@property
def is_internal_router(self) -> bool:
# By default, router/gate is called before FusedMoE forward pass
return self._gate is not None
return self.gate is not None

def _maybe_init_expert_routing_tables(
self,
Expand Down Expand Up @@ -1457,7 +1457,6 @@ def forward_native(
hidden_states: torch.Tensor,
router_logits: torch.Tensor,
) -> torch.Tensor | tuple[torch.Tensor, torch.Tensor]:
self.ensure_moe_quant_config_init()
return self.runner.forward(
hidden_states,
router_logits,
Expand Down
4 changes: 4 additions & 0 deletions vllm/model_executor/layers/fused_moe/runner/default_moe_runner.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ def _moe_forward(
layer_name: str,
) -> torch.Tensor:
layer = get_layer_from_name(layer_name)
# TODO(bnell): this can be removed after MK migration is complete.
layer.ensure_moe_quant_config_init()
return layer.runner.forward_impl(
layer, hidden_states, router_logits, shared_experts_input
)
Expand All @@ -84,6 +86,8 @@ def _moe_forward_shared(
layer_name: str,
) -> tuple[torch.Tensor, torch.Tensor]:
layer = get_layer_from_name(layer_name)
# TODO(bnell): this can be removed after MK migration is complete.
layer.ensure_moe_quant_config_init()
Comment thread
bnellnm marked this conversation as resolved.
return layer.runner.forward_impl(
layer, hidden_states, router_logits, shared_experts_input
)
Expand Down