Skip to content

[Bugfix] fixes wna16 quantization for dense layers. - #35865

Open
sighingnow wants to merge 1 commit into
vllm-project:mainfrom
sighingnow:fixes-wna16-int4
Open

sighingnow wants to merge 1 commit into
vllm-project:mainfrom
sighingnow:fixes-wna16-int4

Conversation

@sighingnow

@sighingnow sighingnow commented Mar 3, 2026

Copy link
Copy Markdown
Collaborator

WNA16 quantization uses GPTQ/AWQ for dense linear layers, which require modules_in_block_to_quantize that resolved from the safetensors metadata to decide which layers are quantization.

However, WNA16 doesn't pass the information to GPTQConfig/AWQConfig when resolving the linear method implementation.

@mergify mergify Bot added qwen Related to Qwen models bug Something isn't working labels Mar 3, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly fixes an issue with WNA16 quantization for dense layers by ensuring modules_in_block_to_quantize is passed to the underlying GPTQ/AWQ configurations. The changes look good and address the described problem. I've added a couple of comments regarding maintainability: one about avoiding an internal API from a dependency, and another about simplifying a redundant dictionary update for better code clarity.

Comment thread vllm/model_executor/layers/quantization/moe_wna16.py
Comment thread vllm/model_executor/models/qwen3_5.py
@mergify

mergify Bot commented Mar 3, 2026

Copy link
Copy Markdown
Contributor

Hi @sighingnow, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?
mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:
# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

@mergify

mergify Bot commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @sighingnow.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Mar 4, 2026
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
@mergify mergify Bot removed the needs-rebase label Mar 5, 2026
@mergify

mergify Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @sighingnow.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@github-actions

Copy link
Copy Markdown

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!

@github-actions github-actions Bot added the stale Over 90 days of inactivity label Aug 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working needs-rebase qwen Related to Qwen models stale Over 90 days of inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant