[Misc][Quantization] Remove kvcache quant weight related patches. by menogrey · Pull Request #11053 · vllm-project/vllm-ascend

menogrey · 2026-06-27T08:21:31Z

What this PR does / why we need it?

After vllm-project/vllm#43167 , we can remove our two patches about kvcache quant weight.
Should be merged after our main2main at vllm-tag 0.23.0 and vllm-main-0605.

Does this PR introduce any user-facing change?

How was this patch tested?

local tested model:
Qwen3-32B-W8A8C8
DeepSeek-V3.1-W4A8C8

vLLM version: v0.23.0
vLLM main: vllm-project/vllm@1f486d9

github-actions · 2026-06-27T08:22:35Z

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

A PR should do only one thing, smaller PRs enable faster reviews.
Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

gemini-code-assist · 2026-06-27T08:25:18Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request cleans up the codebase by removing temporary patches related to KV cache quantization weight loading. The logic for remapping checkpoint weight names has been migrated to a centralized WeightsMapper approach within modelslim_config.py, improving maintainability and reducing the need for intrusive monkey-patching.

Highlights

Removal of Obsolete Patches: Removed patch_weight_utils.py and patch_gqa_c8.py as they are no longer required for KV cache quantization weight loading.
Refactoring Weight Mapping: Introduced get_cache_scale_mapper in modelslim_config.py to centralize weight name remapping logic using WeightsMapper.
Codebase Cleanup: Cleaned up import registrations and references in vllm_ascend/patch/ and vllm_ascend/patch/worker/ to reflect the removal of the patches.

New Features

🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

Suggested PR Title:

[Ops][Misc] Refactor weight loading and remove obsolete patches

Suggested PR Summary:

### What this PR does / why we need it?

This PR removes obsolete weight utility and GQA C8 patches (`patch_weight_utils.py` and `patch_gqa_c8.py`) and refactors the quantization configuration in `modelslim_config.py`. Specifically, it replaces `get_cache_scale` with `get_cache_scale_mapper` to return a `WeightsMapper` for C8, FA, and indexer quantization parameters.

Feedback on this PR includes:
- Renumbering the patch index sections in `vllm_ascend/patch/__init__.py` to maintain a continuous sequence after the removal of sections 21 and 26.
- Using the pre-computed `self.enable_c8_quant` attribute in `modelslim_config.py` instead of directly querying `self.quant_description` to avoid redundant dictionary lookups.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Not specified in the PR.

gemini-code-assist · 2026-06-27T08:27:21Z

-#    Future Plan:
-#       The maybe_remap_kv_scale_name function of the community is reconstructed to support
-#       multiple backends.
 # ** 22. File: worker/patch_v2/patch_input_batch.py**


Since section 21 (worker/patch_weight_utils.py) has been removed, the section numbering is now discontinuous (skipping 21). Please renumber this section to 21 and shift the subsequent sections (22, 23, 24, etc.) accordingly to maintain a consistent and continuous index of patches.

Suggested change

# ** 22. File: worker/patch_v2/patch_input_batch.py**

# ** 21. File: worker/patch_v2/patch_input_batch.py**

gemini-code-assist · 2026-06-27T08:27:21Z

-#       or when the GQA model's weight names are aligned with the parameter
-#       names expected by the quantisation backend.
-# ** 27. File: worker/patch_qwen3vl.py**
+# ** 26. File: worker/patch_qwen3vl.py**


With the removal of section 21 and section 26, this section should be renumbered to 25 instead of 26 to maintain a continuous sequence.

Suggested change

# ** 26. File: worker/patch_qwen3vl.py**

# ** 25. File: worker/patch_qwen3vl.py**

gemini-code-assist · 2026-06-27T08:27:22Z

 #    Future Plan:
 #       Remove this patch when vllm-ascend supports pattern matching for this fused kernel.
-# ** 28. File: worker/patch_qwen3_dflash.py**
+# ** 27. File: worker/patch_qwen3_dflash.py**


Following the renumbering, this section should be renumbered to 26 instead of 27.

Suggested change

# ** 27. File: worker/patch_qwen3_dflash.py**

# ** 26. File: worker/patch_qwen3_dflash.py**

gemini-code-assist · 2026-06-27T08:27:22Z

 #       can be extended by hardware plugins like vllm-ascend.
 #
-# ** 29. File: platform/patch_mamba_manager.py**
+# ** 28. File: platform/patch_mamba_manager.py**


Following the renumbering, this section should be renumbered to 27 instead of 28.

Suggested change

# ** 28. File: platform/patch_mamba_manager.py**

# ** 27. File: platform/patch_mamba_manager.py**

gemini-code-assist · 2026-06-27T08:27:22Z

-        if self.quant_description.get("kv_cache_type") != "C8":
+    def get_cache_scale_mapper(self) -> "WeightsMapper | None":
+        suffix_map = {}
+        if self.quant_description.get("kv_cache_type") == "C8":


Use the pre-computed boolean attribute self.enable_c8_quant instead of querying self.quant_description directly. This is more consistent with how self.enable_fa_quant and self.enable_indexer_quant are used below, and avoids redundant dictionary lookups.

Suggested change

if self.quant_description.get("kv_cache_type") == "C8":

if self.enable_c8_quant:

github-actions · 2026-06-27T08:31:33Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

kunpengW-code · 2026-06-29T10:02:22Z

-#       we have enhanced the maybe_remap_kv_scale_name function.
-#    Future Plan:
-#       The maybe_remap_kv_scale_name function of the community is reconstructed to support
-#       multiple backends.
 # ** 22. File: worker/patch_v2/patch_input_batch.py**


Need to check the serial number after the patch is removed.

In addition to this, LGTM.

Update the serial number

menogrey · 2026-06-30T06:48:49Z

/rerun
[Bot]: rerun completed.

Rerun:

E2E

github-actions · 2026-07-06T06:35:16Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Signed-off-by: menogrey <1299267905@qq.com>

menogrey · 2026-07-07T09:51:03Z

/weekly multi-node-glm4.7-w8a8c8-layerwise
weekly command triggered.

Weekly-A3: workflow run

### What this PR does / why we need it? #11053 introduce a UT error. This PR fix it. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com>

…lm-project#11053) ### What this PR does / why we need it? After vllm-project/vllm#43167 , we can remove our two patches about kvcache quant weight. Should be merged after our main2main at vllm-tag 0.23.0 and vllm-main-0605. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? local tested model: Qwen3-32B-W8A8C8 DeepSeek-V3.1-W4A8C8 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 --------- Signed-off-by: menogrey <1299267905@qq.com>

### What this PR does / why we need it? vllm-project#11053 introduce a UT error. This PR fix it. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com>

Signed-off-by: menogrey <1299267905@qq.com>

…hes." (#11615) Reverts #11053 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com>

…lm-project#11053) ### What this PR does / why we need it? After vllm-project/vllm#43167 , we can remove our two patches about kvcache quant weight. Should be merged after our main2main at vllm-tag 0.23.0 and vllm-main-0605. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? local tested model: Qwen3-32B-W8A8C8 DeepSeek-V3.1-W4A8C8 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 --------- Signed-off-by: menogrey <1299267905@qq.com>

…lm-project#11053) ### What this PR does / why we need it? After vllm-project/vllm#43167 , we can remove our two patches about kvcache quant weight. Should be merged after our main2main at vllm-tag 0.23.0 and vllm-main-0605. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? local tested model: Qwen3-32B-W8A8C8 DeepSeek-V3.1-W4A8C8 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 --------- Signed-off-by: menogrey <1299267905@qq.com> Signed-off-by: Spicy-Stick <873805887@qq.com>

### What this PR does / why we need it? vllm-project#11053 introduce a UT error. This PR fix it. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com> Signed-off-by: Spicy-Stick <873805887@qq.com>

…hes." (#11615) Reverts vllm-project/vllm-ascend#11053 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com> Signed-off-by: xqchen7 <chenxueqing7@huawei.com>

…lm-project#11053) ### What this PR does / why we need it? After vllm-project/vllm#43167 , we can remove our two patches about kvcache quant weight. Should be merged after our main2main at vllm-tag 0.23.0 and vllm-main-0605. ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? local tested model: Qwen3-32B-W8A8C8 DeepSeek-V3.1-W4A8C8 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 --------- Signed-off-by: menogrey <1299267905@qq.com>

### What this PR does / why we need it? vllm-project#11053 introduce a UT error. This PR fix it. - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com>

…hes." (vllm-project#11615) Reverts vllm-project#11053 - vLLM version: v0.23.0 - vLLM main: vllm-project/vllm@1f486d9 Signed-off-by: menogrey <1299267905@qq.com>

menogrey requested a review from wangxiyuan as a code owner June 27, 2026 08:21

github-actions Bot added the module:quantization label Jun 27, 2026

gemini-code-assist Bot reviewed Jun 27, 2026

View reviewed changes

github-actions Bot added the merge-conflicts label Jun 27, 2026

kunpengW-code reviewed Jun 29, 2026

View reviewed changes

menogrey force-pushed the remove_kvcache_patch branch from a8aff3e to 239e48c Compare June 30, 2026 06:42

menogrey changed the title ~~[Quantization] Remove kvcache quant weight related patches.~~ [Misc][Quantization] Remove kvcache quant weight related patches. Jun 30, 2026

github-actions Bot added merge-conflicts and removed merge-conflicts labels Jun 30, 2026

menogrey added 3 commits July 7, 2026 14:35

Remove kvcache weight related patch.

9bea950

Signed-off-by: menogrey <1299267905@qq.com>

Fix the serial number.

3b68f39

Signed-off-by: menogrey <1299267905@qq.com>

Fix bug and comment.

6b524aa

Signed-off-by: menogrey <1299267905@qq.com>

menogrey force-pushed the remove_kvcache_patch branch from dbca1aa to 6b524aa Compare July 7, 2026 06:51

github-actions Bot removed the merge-conflicts label Jul 7, 2026

Fix indexer mapping.

cae4b20

Signed-off-by: menogrey <1299267905@qq.com>

kunpengW-code reviewed Jul 7, 2026

View reviewed changes

Comment thread vllm_ascend/patch/worker/__init__.py

Fix delete.

aa4108f

Signed-off-by: menogrey <1299267905@qq.com>

zzzzwwjj approved these changes Jul 7, 2026

View reviewed changes

kunpengW-code approved these changes Jul 7, 2026

View reviewed changes

zzzzwwjj approved these changes Jul 7, 2026

View reviewed changes

zzzzwwjj merged commit 918f6dc into vllm-project:main Jul 7, 2026
7 checks passed

menogrey mentioned this pull request Jul 7, 2026

[Misc] Fix UT error. #11581

Merged

menogrey mentioned this pull request Jul 8, 2026

Revert "[Misc][Quantization] Remove kvcache quant weight related patches." #11615

Merged

menogrey added a commit to menogrey/vllm-ascend that referenced this pull request Jul 8, 2026

Revert vllm-project#11053.

888ff7f

Signed-off-by: menogrey <1299267905@qq.com>

menogrey mentioned this pull request Jul 8, 2026

[Misc] fix ci error #11607

Closed

	# 22. File: worker/patch_v2/patch_input_batch.py
	# 21. File: worker/patch_v2/patch_input_batch.py

	# 26. File: worker/patch_qwen3vl.py
	# 25. File: worker/patch_qwen3vl.py

	# 27. File: worker/patch_qwen3_dflash.py
	# 26. File: worker/patch_qwen3_dflash.py

	# 28. File: platform/patch_mamba_manager.py
	# 27. File: platform/patch_mamba_manager.py

	if self.quant_description.get("kv_cache_type") == "C8":
	if self.enable_c8_quant:

Uh oh!

Conversation

menogrey commented Jun 27, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

github-actions Bot commented Jun 27, 2026

Uh oh!

gemini-code-assist Bot commented Jun 27, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot Jun 27, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Jun 27, 2026

Uh oh!

kunpengW-code Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

kunpengW-code Jun 29, 2026

Choose a reason for hiding this comment

Uh oh!

menogrey Jul 7, 2026

Choose a reason for hiding this comment

Uh oh!

menogrey commented Jun 30, 2026 • edited by vllm-ascend-ci Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 6, 2026

Uh oh!

Uh oh!

menogrey commented Jul 7, 2026 • edited by vllm-ascend-ci Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

menogrey commented Jun 27, 2026 •

edited by github-actions Bot

Loading

menogrey commented Jun 30, 2026 •

edited by vllm-ascend-ci

Loading

menogrey commented Jul 7, 2026 •

edited by vllm-ascend-ci

Loading