fix(gguf): Make GGUFMoEMethod.apply() parameters optional#30423
fix(gguf): Make GGUFMoEMethod.apply() parameters optional#30423kitaekatt wants to merge 1 commit intovllm-project:mainfrom
Conversation
The GGUFMoEMethod.apply() method had required positional parameters (top_k, renormalize) that were not being passed by the caller in FusedMoE.forward_impl(). The base class FusedMoEMethodBase.apply() only requires (layer, x, router_logits). These parameters were also not used in the method body - it calls layer.select_experts() without passing them. This change makes top_k and renormalize keyword arguments with defaults, matching how the caller invokes the method. Fixes GGUF MoE models (e.g., Qwen3-30B-A3B-Instruct-GGUF) failing with: TypeError: GGUFMoEMethod.apply() missing 2 required positional arguments: 'top_k' and 'renormalize'
|
This pull request has merge conflicts that must be resolved before it can be |
There was a problem hiding this comment.
Code Review
This pull request correctly addresses a TypeError in GGUFMoEMethod.apply() by making the top_k and renormalize parameters optional. According to the pull request description, these parameters were required but not always passed by the caller, and as they are not used within the method body, making them optional with default values is a safe and effective fix. The change is minimal and directly resolves the bug affecting GGUF MoE models. The implementation looks good.
|
Closing this PR as it's no longer needed. Upstream PR #29066 ( Thanks to the vLLM team for the comprehensive MoE refactor! |
Summary
top_kandrenormalizekeyword arguments with defaults inGGUFMoEMethod.apply()Problem
GGUF MoE models (e.g., Qwen3-30B-A3B-Instruct-GGUF) fail with:
Root Cause
FusedMoE.forward_impl()inlayer.py:1946callsquant_method.apply(layer=self, x=..., router_logits=...)FusedMoEMethodBase.apply()only requires(layer, x, router_logits)GGUFMoEMethod.apply()had extra required positional args that weren't being passedlayer.select_experts()without them)Test plan
🤖 Generated with Claude Code