[Misc][Quantization] Remove kvcache quant weight related patches. - #12463
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request streamlines the KV cache quantization weight loading process by replacing legacy monkey-patches with a structured WeightsMapper implementation. This change improves maintainability, reduces code complexity, and aligns the quantization configuration with standard vLLM patterns. Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Ops][Misc] Refactor cache scale mapping to use WeightsMapper and remove obsolete patchesSuggested PR Summary:
### What this PR does / why we need it?
This PR refactors the cache scale mapping mechanism in `AscendModelSlimConfig` by replacing the custom `get_cache_scale` method with `get_cache_scale_mapper()`, which returns a `WeightsMapper`. This aligns with upstream vLLM's design.
As a result, the custom patches `patch_weight_utils.py` and `patch_gqa_c8.py` (which were previously used to intercept and remap C8/FA/indexer quantization scales) are no longer needed and have been removed.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added unit tests in `tests/ut/quantization/test_modelslim_config.py` covering:
- Default mapper behavior
- C8 KV cache type mapper
- FA quantization mapper
- Indexer quantization mapperI have no additional feedback to provide as there are no review comments.
Signed-off-by: menogrey <1299267905@qq.com>
|
/rerun Rerun:
|
|
/weekly multi-node-glm4.7-w8a8c8-layerwise
|
What this PR does / why we need it?
After vllm-project/vllm#43167 vllm-project/vllm#44589, we can remove our two patches about kvcache quant weight.
Does this PR introduce any user-facing change?
How was this patch tested?