[Bugfix] fixes wna16 quantization for dense layers. - #35865
sighingnow wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request correctly fixes an issue with WNA16 quantization for dense layers by ensuring modules_in_block_to_quantize is passed to the underlying GPTQ/AWQ configurations. The changes look good and address the described problem. I've added a couple of comments regarding maintainability: one about avoiding an internal API from a dependency, and another about simplifying a redundant dictionary update for better code clarity.
|
Hi @sighingnow, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Tao He <linzhu.ht@alibaba-inc.com>
b5b0610 to
5a2cf83
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
|
This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you! |
WNA16 quantization uses GPTQ/AWQ for dense linear layers, which require
modules_in_block_to_quantizethat resolved from the safetensors metadata to decide which layers are quantization.However, WNA16 doesn't pass the information to
GPTQConfig/AWQConfigwhen resolving the linear method implementation.