[quant][autoround]Refactor INC quantization into package with INCScheme orchestrator - #40601
Merged
Merged
Conversation
yiliu30
requested review from
mgoin,
pavanimajety,
robertgshaw2-redhat,
tlrmchlsmth and
yewentao256
as code owners
April 22, 2026 08:43
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
Hi @yiliu30, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request refactors the Intel Neural Compressor (INC) quantization support into a modular, scheme-based architecture. It introduces a new directory structure, a configuration resolver, and a dispatch system for different quantization schemes like WNA16 across CPU and XPU platforms. Feedback focuses on correcting the handling of unquantized or unsupported FusedMoE layers, which currently return incompatible methods or None, potentially leading to runtime errors.
Contributor
Author
|
Hi @jikunshang @xinyu-intel @wenhuach21 @Zhenzhong1 please take a look when you have time, thanks! |
Contributor
Author
|
@claude review |
1 task
Contributor
|
Hi @yiliu30, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
2 similar comments
Contributor
|
Hi @yiliu30, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Contributor
|
Hi @yiliu30, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
yiliu30
changed the title
4 tasks
jikunshang
added
intel-gpu
yiliu30
requested review from
ApostaC,
ZJY0516,
gshtras,
orozery,
sighingnow,
tdoublep,
tjtanaa,
vadiklyutiy and
zyongye
as code owners
May 11, 2026 05:50
Signed-off-by: Zhenzhong Xu <zhenzhong.xu@intel.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Signed-off-by: yiliu30 <yi4.liu@intel.com>
Contributor
Author
Hi @jikunshang, the issue has been resolved by upgrading the auto-round-lib — thanks for your patience! Please take a look again when you get a chance. |
jikunshang
reviewed
Jun 16, 2026
| @@ -25,8 +48,720 @@ | |||
Signed-off-by: yiliu30 <yi4.liu@intel.com>
jikunshang
approved these changes
Jun 17, 2026
|
I'm exterimenting with main branches (because i've got Intel Arc B70 and it's my only option to build everything from master) and after this PR vllm is failing here command I use to run is following with and without XPU_GRAPH, with cache fp8 and turboquant_k8v4 I understand it's rc-dev so I have zero expectations, just wanted to share some data and experience |
Contributor
Author
|
Hi @urakozz, thanks for raising this issue. Could you please check the auto-round-lib version? This requires |
Contributor
I don't see any version check in the code. If that's the case, we should add one. |
in requirements/xpu.txt it's UPD: i just noticed that it's about patch version .3, will try with full log: |
Contributor
Author
|
Hi @urakozz, thanks for providing the detailed log. We recently added the new auto-round-lib WOQ kernel while keeping the previous vllm-xpu-kernels path. When auto-round-lib is installed, vLLM currently selects the auto-round-lib WOQ kernel by default. From the log, it looks like the model is dispatched to the auto-round-lib kernel. This WOQ kernel + compile path is not fully supported yet, but we plan to make it compatible in the near future. For now, you could try running with BTW, the latest auto-round-lib version is |
|
So, i have an update. I rebased onto the laterst master, updated auto-round-lib to 0.13.4 and it was still crashing. so what i did currently and it helps in my case:
I added flag to force use |
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Follow the compressed-tensor style by replacing the monolithic inc.py with an inc/ package that uses scheme-based dispatch, paving the way for adding more schemes in the near future.
Part of #37979
Test Plan
Local test models
cc @hshen14 @thuang6