[EPLB] Avoiding eplb's dependency on a specified model by shenchuxiaofugui · Pull Request #6528 · vllm-project/vllm-ascend

shenchuxiaofugui · 2026-02-04T03:46:47Z

What this PR does / why we need it?

Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted.
Add some log about eplb.

Does this PR introduce any user-facing change?

How was this patch tested?

Deepseek v3.1 chat

Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like "has pointy ears, whiskers, and a tail." Instead, you show them many pictures, saying "this is a cat" or "this is not a cat." The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define "cat-ness."\n\nDeep learning is essentially this, but for computers. It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\nDeep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI). It uses artificial neural networks with many layers ("deep" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. Artificial Intelligence (AI): The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. Machine Learning (ML): A subset of AI that gives computers the ability to learn from data without being explicitly programmed for every single rule.\n3. Deep Learning (DL): A specific, powerful

vLLM version: v0.15.0
vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0

gemini-code-assist · 2026-02-04T03:47:02Z

Summary of Changes

Hello @shenchuxiaofugui, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request streamlines the EPLB (Expert Parallelism Load Balancing) component by removing unused code and unnecessary model-specific dependencies. It eliminates an abstract adaptor class and associated inheritance, along with model-type-specific attribute registration that was not actively used. The overall impact is a cleaner, more generic, and less coupled EPLB implementation.

Highlights

Removed unused abstract adaptor: The vllm_ascend/eplb/adaptor/abstract_adaptor.py file, which defined the EplbAdaptor abstract base class, has been entirely removed as it was no longer needed.
Decoupled VllmEplbAdaptor: The VllmEplbAdaptor class in vllm_ascend/eplb/adaptor/vllm_adaptor.py no longer inherits from the now-removed EplbAdaptor, simplifying its class hierarchy.
Eliminated model-specific attribute registration: The logic in vllm_ascend/eplb/utils.py that registered specific attributes (like num_moe_layers and num_dense_layers) based on the model type (e.g., qwen3_moe, deepseek_v2, deepseek_v3) has been removed, as these attributes were not being utilized.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

vllm_ascend/eplb/adaptor/abstract_adaptor.py
- Removed the entire file, including the EplbAdaptor abstract base class.
vllm_ascend/eplb/adaptor/vllm_adaptor.py
- Removed the import statement for EplbAdaptor.
- Updated VllmEplbAdaptor to no longer inherit from EplbAdaptor.
vllm_ascend/eplb/utils.py
- Deleted the code block responsible for registering model-specific attributes like num_moe_layers and num_dense_layers based on model_type.
- Removed the NotImplementedError that was raised for unsupported model types in the EPLB registration logic.

Activity

No human activity (comments, reviews, etc.) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-02-04T03:48:15Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist

Code Review

This pull request aims to make the Expert Parallelism Load Balancer (EPLB) more model-agnostic by removing an abstract adaptor class and model-specific attribute registration. While the goal of decoupling is good, the current implementation introduces two critical issues. First, a super().__init__ call in VllmEplbAdaptor becomes problematic after its base class is removed, potentially causing a TypeError at runtime. Second, the removal of model attribute initializations in utils.py breaks other parts of the code that depend on these attributes, which will lead to an AttributeError. Both issues are critical and need to be addressed before this PR can be merged.

I am having trouble creating individual review comments. Click here to see my feedback.

vllm_ascend/eplb/adaptor/vllm_adaptor.py (28)

With EplbAdaptor removed as a base class, VllmEplbAdaptor now implicitly inherits from object. The call super().__init__(**args) will thus invoke object.__init__(**args). Since object.__init__ takes no arguments, this will raise a TypeError at runtime if VllmEplbAdaptor is instantiated with any keyword arguments. This line should be removed as it serves no purpose and introduces a potential crash.

vllm_ascend/eplb/utils.py (66-74)

The removal of this code block, which sets model.num_moe_layers and model.num_dense_layers, will cause runtime errors. Methods like get_all_moe_loads and clear_all_moe_loads are patched onto the model object in this same file and rely on these attributes. For example, get_all_moe_loads uses self.num_moe_layers (where self is the model object). Without these attributes being set on the model, an AttributeError will be raised when these methods are called. While the values are now computed in VllmEplbAdaptor, they are not accessible from the methods patched on the model. This change breaks existing functionality and should be reverted, or the dependent methods must be refactored.

whx-sjtu

The refactor looks good to me. But the usage part seems confused to me. Please decide whether to use environmental variable or config parameter @zzzzwwjj.

Signed-off-by: shenchuxiaofugui <1311027364@qq.com>

…to qwen3next_rebase * 'main' of https://github.com/vllm-project/vllm-ascend: [Feat] 310p support MoE W8A8 quantizaition (vllm-project#6641) [TEST]add a qwen3-30b acc case with mooncake mempool (vllm-project#6244) [MOE Refactor] Remove QuantType in prepare_finalize.py (vllm-project#6534) [EPLB] Avoiding eplb's dependency on a specified model (vllm-project#6528) [Doc][Misc] Restructure tutorial documentation (vllm-project#6501) implement batch invariant with ascendc (vllm-project#6590) [Refact]Refact MLA/SFA weight prefetch to consist with moe weight prefetch (vllm-project#6629) [Misc] upgrade to vllm main (vllm-project#6646) [main][Docs] Fix spelling errors across documentation (vllm-project#6649) [bugfix]Fix no attribute 'data' when MLAPO is enable (vllm-project#6601) [DOC]Add Memcache Usage Guide (vllm-project#6476) [main][bugfix] Fix spec acceptance rate problem in vllm_0.15.0 (vllm-project#6606) [Test][LoRA] Add e2e test for base model inference (vllm-project#6624) [refactor]Optimized the kvcache usage of Deepseek v3.2 (vllm-project#6610) [Feat](sfa,dcp) support dcp for sfa (vllm-project#6563) [BugFix] Add support for rotary_dim parameter when using partial rope in rotary_embedding (vllm-project#6581) [fix bug] fix tensor mismatch bug in sigmoid operate test case (vllm-project#6619) [Kernel]: Optimize DispatchFFNCombine performance (vllm-project#6468) [MISC] Clean up useless env USE_OPTIMIZED_MODEL (vllm-project#6618)

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com> Signed-off-by: momochenchuw <chenchuw@huawei.com>

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com>

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com> Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com>

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com> Signed-off-by: zrj026 <zhangrunjiang026@gmail.com>

…6528) ### What this PR does / why we need it? 1. Currently, eplb registers different attributes for different models, but these attributes are not actually used. Now, these attributes are directly deleted. 2. Add some log about eplb. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? #### Deepseek v3.1 chat Of course! Here is a comprehensive explanation of deep learning, broken down for clarity.\n\n### The Simple Analogy: A Child Learning to Recognize a Cat\n\nImagine teaching a child what a cat is. You don't give them a rulebook with instructions like \"has pointy ears, whiskers, and a tail.\" Instead, you show them many pictures, saying \"this is a cat\" or \"this is not a cat.\" The child's brain gradually learns to identify the complex patterns—the combination of shapes, colors, and textures—that define \"cat-ness.\"\n\n**Deep learning is essentially this, but for computers.** It's a method for teaching computers to learn from examples and recognize patterns directly from data (like images, sound, or text) without being explicitly programmed with rigid rules.\n\n---\n\n### The Technical Definition\n\n**Deep Learning is a subfield of machine learning, which itself is a subfield of artificial intelligence (AI).** It uses artificial **neural networks** with many layers (\"deep\" networks) to model and understand complex patterns in data.\n\nHere are the key concepts in that definition:\n\n1. **Artificial Intelligence (AI):** The broad science of making machines smart and capable of performing tasks that typically require human intelligence.\n2. **Machine Learning (ML):** A subset of AI that gives computers the ability to learn from data *without* being explicitly programmed for every single rule.\n3. **Deep Learning (DL):** A specific, powerful - vLLM version: v0.15.0 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.15.0 Signed-off-by: shenchuxiaofugui <1311027364@qq.com>

shenchuxiaofugui requested a review from wangxiyuan as a code owner February 4, 2026 03:46

gemini-code-assist bot reviewed Feb 4, 2026

View reviewed changes

shenchuxiaofugui force-pushed the del branch 2 times, most recently from 25806b1 to 139b015 Compare February 4, 2026 06:49

shenchuxiaofugui requested a review from MengqingCao as a code owner February 4, 2026 06:49

shenchuxiaofugui force-pushed the del branch from 139b015 to dcbbd83 Compare February 4, 2026 07:36

weijinqian0 approved these changes Feb 4, 2026

View reviewed changes

weijinqian0 self-requested a review February 4, 2026 08:43

whx-sjtu added ready read for review ready-for-test start test by label for PR labels Feb 4, 2026

whx-sjtu requested a review from zzzzwwjj February 4, 2026 08:52

whx-sjtu approved these changes Feb 4, 2026

View reviewed changes

shenchuxiaofugui force-pushed the del branch from dcbbd83 to f0251d2 Compare February 4, 2026 09:04

weijinqian0 approved these changes Feb 4, 2026

View reviewed changes

shenchuxiaofugui force-pushed the del branch from f0251d2 to 57a7cce Compare February 5, 2026 07:08

[EPLB] Avoiding eplb's dependency on a specified model

ef438a4

Signed-off-by: shenchuxiaofugui <1311027364@qq.com>

shenchuxiaofugui force-pushed the del branch from 57a7cce to ef438a4 Compare February 6, 2026 02:01

shenchuxiaofugui mentioned this pull request Feb 9, 2026

[RFC]: Expert Parallelism Load Balancer #5633

Open

18 tasks

weijinqian0 approved these changes Feb 9, 2026

View reviewed changes

wangxiyuan approved these changes Feb 10, 2026

View reviewed changes

wangxiyuan merged commit 34eecac into vllm-project:main Feb 10, 2026
33 of 36 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[EPLB] Avoiding eplb's dependency on a specified model#6528

[EPLB] Avoiding eplb's dependency on a specified model#6528
wangxiyuan merged 1 commit intovllm-project:mainfrom
shenchuxiaofugui:del

shenchuxiaofugui commented Feb 4, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Feb 4, 2026

Uh oh!

github-actions bot commented Feb 4, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

whx-sjtu left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

shenchuxiaofugui commented Feb 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Deepseek v3.1 chat

Uh oh!

gemini-code-assist bot commented Feb 4, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions bot commented Feb 4, 2026

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

vllm_ascend/eplb/adaptor/vllm_adaptor.py (28)

vllm_ascend/eplb/utils.py (66-74)

Uh oh!

whx-sjtu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

shenchuxiaofugui commented Feb 4, 2026 •

edited

Loading