add 2/3/5/6/7 CUDA support in AutoRound format

[wenhuach21]

Purpose

Restore CUDA support for 2-bit and 3-bit AutoRound formats

Test Plan

https://huggingface.co/Intel/Qwen3.8-27B-bpw2.8-AutoRound

Test Result

pass

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

cc @jikunshang

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[wenhuach21]

please have a review when you are free

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @wenhuach21.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[wenhuach21]

/ci run

[github-actions]

✅ Triggered Buildkite CI #86988 for commit d3296c1dc0d2.

[wenhuach21]

/ci retry

[github-actions]

✅ Queued 13 failed job(s) for retry in Buildkite CI #86988.

[wenhuach21]

/ci retry

[github-actions]

✅ Queued 13 failed job(s) for retry in Buildkite CI #86988.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 0c5fbeec-d381-4497-bd14-9c8ad1b3e439

📥 Commits

Reviewing files that changed from the base of the PR and between d6bce42 and 0f6d4c6.

📒 Files selected for processing (3)

• tests/quantization/test_auto_round.py
• vllm/model_executor/layers/quantization/inc/inc.py
• vllm/model_executor/layers/quantization/inc/schemes/inc_wna16_scheme.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Summary

Summary by CodeRabbit

• New Features

  • Added CUDA support for 2-, 3-, 5-, 6-, and 7-bit WNA16 quantization using optimized Humming kernels.
  • Extended supported INC quantization widths to include 5-, 6-, and 7-bit configurations.
  • Added support for low-bit GPTQ/AWQ linear and mixture-of-experts layers.

• Tests

  • Added coverage verifying correct CUDA routing for low-bit configurations and preserving existing paths for 4- and 8-bit layers.

Walkthrough

The change expands INC bit-width declarations and routes selected CUDA WNA16 linear and MoE configurations to Humming. CUDA tests cover Humming dispatch for 2- and 3-bit layers and existing INC dispatch for 4- and 8-bit layers.

Changes

CUDA WNA16 Humming routing

Layer / File(s)	Summary
Humming configuration construction vllm/model_executor/layers/quantization/inc/inc.py, vllm/model_executor/layers/quantization/inc/schemes/inc_wna16_scheme.py	INCConfig adds 5-, 6-, and 7-bit support. New helpers construct Humming quantization, linear, and MoE methods from GPTQ or AWQ schemas.
CUDA low-bit dispatch vllm/model_executor/layers/quantization/inc/schemes/inc_wna16_scheme.py	CUDA WNA16 linear and MoE layers with 2-, 3-, 5-, 6-, or 7-bit weights route to Humming.
Routing regression coverage tests/quantization/test_auto_round.py	CUDA tests verify Humming routing for 2- and 3-bit layers and preserve INCLinearMethod routing for 4- and 8-bit linear layers.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 0f6d4

This change enables CUDA Humming routing for supported low-bit AutoRound formats while retaining existing 4- and 8-bit linear behavior. Current coverage supports merge readiness with no active merge-blocking risk.

Suggested reviewers: zhenzhong1

Sequence Diagram(s)

sequenceDiagram
  participant LayerConfig
  participant INCWNA16LinearScheme
  participant HummingMethod
  LayerConfig->>INCWNA16LinearScheme: provide CUDA WNA16 bit width
  INCWNA16LinearScheme->>HummingMethod: build Humming method for supported low-bit width

Loading

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 12 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: adding CUDA support for 2-, 3-, 5-, 6-, and 7-bit AutoRound formats.
Description check	✅ Passed	The description is related to the changeset. It states the purpose, identifies the test plan, and reports a passing result.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[wenhuach21]

/ci run

[github-actions]

✅ Triggered Buildkite CI #87195 for commit 0f6d4c62ce38.