[Misc][Refactor] Separate router from FusedMoE class#30573
[Misc][Refactor] Separate router from FusedMoE class#30573bnellnm wants to merge 8 commits intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the Mixture-of-Experts (MoE) routing logic by introducing an abstract FusedMoERouter class and a concrete DefaultFusedMoERouter implementation. The FusedMoERouter encapsulates all expert selection parameters and methods, which were previously part of the FusedMoE layer. The FusedMoE layer now instantiates a DefaultFusedMoERouter and delegates expert selection to it. Consequently, various quantization methods' apply functions have been updated to accept and utilize this new router object for expert selection. Additionally, EPLB (Expert Placement Load Balancing) state variables have been consolidated into a new EplbLayerState dataclass. A review comment highlights a RecursionError in the DefaultFusedMoERouter's routing_method_type property, suggesting it should return the private _routing_method_type attribute instead of calling itself.
vllm/model_executor/layers/fused_moe/default_fused_moe_router.py
Outdated
Show resolved
Hide resolved
|
This pull request has merge conflicts that must be resolved before it can be |
63afe60 to
15482a0
Compare
15482a0 to
b95e81f
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
b95e81f to
bb81318
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
bb81318 to
486efb8
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
Purpose
Separate existing router logic into a standalone class
DefaultFusedMoERouter.Needs #30519
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.