Skip to content

[Misc][Refactor] Separate router from FusedMoE class#30573

Closed
bnellnm wants to merge 8 commits intovllm-project:mainfrom
neuralmagic:fused-moe-router-2
Closed

[Misc][Refactor] Separate router from FusedMoE class#30573
bnellnm wants to merge 8 commits intovllm-project:mainfrom
neuralmagic:fused-moe-router-2

Conversation

@bnellnm
Copy link
Collaborator

@bnellnm bnellnm commented Dec 12, 2025

Purpose

Separate existing router logic into a standalone class DefaultFusedMoERouter.

Needs #30519

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the Mixture-of-Experts (MoE) routing logic by introducing an abstract FusedMoERouter class and a concrete DefaultFusedMoERouter implementation. The FusedMoERouter encapsulates all expert selection parameters and methods, which were previously part of the FusedMoE layer. The FusedMoE layer now instantiates a DefaultFusedMoERouter and delegates expert selection to it. Consequently, various quantization methods' apply functions have been updated to accept and utilize this new router object for expert selection. Additionally, EPLB (Expert Placement Load Balancing) state variables have been consolidated into a new EplbLayerState dataclass. A review comment highlights a RecursionError in the DefaultFusedMoERouter's routing_method_type property, suggesting it should return the private _routing_method_type attribute instead of calling itself.

@mergify
Copy link

mergify bot commented Dec 18, 2025

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bnellnm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Dec 18, 2025
@bnellnm bnellnm force-pushed the fused-moe-router-2 branch from 63afe60 to 15482a0 Compare January 6, 2026 17:45
@mergify mergify bot removed the needs-rebase label Jan 6, 2026
@bnellnm bnellnm force-pushed the fused-moe-router-2 branch from 15482a0 to b95e81f Compare January 6, 2026 17:57
@mergify
Copy link

mergify bot commented Jan 6, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bnellnm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify
Copy link

mergify bot commented Jan 7, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bnellnm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Jan 7, 2026
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
Signed-off-by: Bill Nell <bnell@redhat.com>
@bnellnm bnellnm force-pushed the fused-moe-router-2 branch from bb81318 to 486efb8 Compare January 8, 2026 18:14
@mergify mergify bot removed the needs-rebase label Jan 8, 2026
@github-project-automation github-project-automation bot moved this to Backlog in MoE Refactor Jan 9, 2026
@robertgshaw2-redhat robertgshaw2-redhat moved this from Backlog to In progress in MoE Refactor Jan 9, 2026
@robertgshaw2-redhat robertgshaw2-redhat moved this from In progress to Backlog in MoE Refactor Jan 9, 2026
@mergify
Copy link

mergify bot commented Jan 9, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @bnellnm.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify bot added the needs-rebase label Jan 9, 2026
@bnellnm bnellnm closed this Jan 10, 2026
@github-project-automation github-project-automation bot moved this from Backlog to Done in MoE Refactor Jan 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants